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MEASURING  OFFICER  KNOWLEDGE  AND  EXPERIENCE  TO  ENABLE  TAILORED 
TRAINING 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  U.S.  Army  requires  effective  and  efficient  training.  However,  what  is  effective  and 
efficient  varies  from  group  to  group  and  individual  to  individual.  For  decades  researchers  have 
explored  the  extent  to  which  training  efficiency/effectiveness  can  be  improved  by  tailoring 
training,  that  is,  by  assessing  salient  individual  differences  and  assigning  learners  to  learning 
conditions  based  on  those  differences.  Criterion-relevant  experience  and  prior  knowledge  are 
arguably  the  most  robust  predictors  of  performance,  and  therefore  are  viable  bases  for  tailoring 
training.  On  this  basis,  effective  and  efficient  measures  of  experience  and  prior  knowledge 
should  be  developed  and  empirically  validated.  Such  valid  measures  are  required  if  instructors 
are  to  adapt  training  to  individuals  who  vary  in  prior  knowledge  and  experience. 

Procedure: 

Instructors  from  the  Engineer  Captains  Career  Course  (ECCC)  at  Fort  Leonard  Wood, 
MO,  were  interviewed  to  determine  which  parts  of  the  course  could  best  distinguish  the 
performance  of  different  officers.  Based  on  those  interviews,  performance  on  the  Defensive 
Planning  exam  was  chosen,  as  it  clearly  indicated  some  officers  as  performing  well,  some 
average,  and  some  poorly.  Five  types  of  predictors  were  constructed  to  assess  how  they  were 
related  to  how  well  officers  performed  on  the  Defensive  Planning  exam.  The  first  was  small  group 
instructors’  (SGI)  forecasts  of  officers’  later  performance  on  their  Defensive  Planning  exams.  The 
second  was  general  biographic  characteristics  of  the  officers,  which  anecdotal  evidence  indicated 
instructors  used  to  assess  relevant  experience.  The  third  was  officers’  scores  on  a  measure  that 
asked  questions  relevant  to  their  Defensive  Planning  training  and  educational  experiences.  The 
fourth  asked  officers  to  rate  their  own  ability  to  execute  activities  related  to  Defensive  Planning. 
The  fifth  type  was  a  test  of  prior  knowledge.  The  instruments  were  reviewed  and  approved  by  the 
instructors,  and  the  final  version  was  administered  to  5  SGIs  and  78  students. 

Findings: 

Analyses  revealed  a  complex  relationship  between  officers’  prior  enlistment  experience 
and  prior  knowledge,  and  their  performance  on  the  Defensive  Planning  exam.  For  officers  with 
prior  enlistment  experience,  there  were  no  significant  predictors  of  exam  performance.  For 
officers  without  prior  enlistment  experience,  prior  knowledge  alone  was  a  significant  predictor. 
Analyses  also  showed  that  for  officers  without  prior  enlistment  experience,  the  predictors  and 
criterion  were  systematically  related  in  a  way  strikingly  similar  to  that  seen  in  the  occupational 
research  literature.  This  similarity  did  not  hold  for  the  prior-enlisted  officers,  however.  We 
discussed  in  detail  how  these  findings  provide  information  which  could  be  used  to  enable  course 
personnel  to  make  informed  decisions  about  tailoring  training. 
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Utilization  and  Dissemination  of  Findings: 


The  findings  demonstrate  the  utility  of  prior  knowledge  measures  for  predicting 
performance  and  thus  infonning  subsequent  implementation  of  tailored  training.  These  findings 
have  been  disseminated  to  Engineer  Captains  Career  Course  instructors  at  Fort  Leonard  Wood, 
MO  and  briefed  to  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  personnel  at  Fort 
Eustis,  VA. 
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TRAINING 


Introduction 

Operational  tempo  requires  U.S.  Anny  personnel  to  learn  more  in  less  time,  and  thereby 
highlights  the  need  for  effective  and  efficient  training.  Given  that  fact,  we  also  know  that  there 
is  great  diversity  in  the  Anny  in  tenns  of  individual  learning  differences  related  to  learning 
ability,  learning  preferences,  prior  knowledge,  prior  experiences,  etc.  There  is  ample  evidence  to 
suggest  that  learning-related  individual  differences  exist  (Jensen,  1998;  Thorndike,  1985)  and 
that  these  individual  differences  interact  with  learning  conditions  (McNamara,  Kintsch,  Songer, 
&  Kintsch,  1996).  That  is,  a  given  approach  to  training  may  be  effective  and  efficient  for  one 
type  of  learner  but  not  another. 

This  suggests  the  need  for  tailored  training.  The  central  idea  to  tailored  training  is  that  it 
is  possible  to  assess  salient  individual  differences  and  then  assign  learners  to  learning  conditions 
based  on  those  differences.  For  example,  individuals  high  in  prior  domain  knowledge  do  better 
when  textbooks  reserve  explanations  for  more  advanced  concepts.  Conversely,  individuals  low 
in  prior  domain  knowledge  do  better  when  explanations  are  given  for  easy  concepts  as  well 
(McNamara,  Kintsch,  Songer,  &  Kintsch,  1996). 

For  such  tailored  training  to  be  effective,  at  least  two  conditions  must  be  satisfied.  First, 
there  must  be  evidence  demonstrating  a  significant  relationship  between  one  or  more  individual 
differences  and  performance.  Second,  there  must  be  evidence  of  an  interaction  between  one  or 
more  individual  differences  and  the  training  condition  (Pashler,  McDaniel,  Doug,  &  Bjorn, 
2009).  Returning  to  the  textbook  example  of  McNamara  et  ah,  the  first  condition  is  met  by  the 
existence  of  a  significant  relationship  between  a  prior  knowledge  test  and  a  test  on  the  textbook 
content.  The  second  condition  is  met  by  the  fact  that  prior  knowledge  interacts  with  type  of 
textbook.  Namely,  high  prior  knowledge  individuals  perform  better  with  the  textbook  which 
explains  only  advanced  concepts,  while  low  prior  knowledge  individuals  perform  better  with  the 
textbook  which  explains  the  simple  concepts  as  well. 

The  goal  of  this  research  was  to  satisfy  the  first  condition — to  develop  and  empirically 
validate  one  or  more  individual  difference  measures  that  are  significantly  related  to  criterion 
performance  in  an  officer  course.  After  we  assessed  the  evidence  for  such  a  relationship,  we 
explored  how  the  predictor  information  might  be  used  to  assign  individuals  to  different  training 
conditions. 

We  chose  to  focus  primarily  on  the  individual  difference  of  prior  knowledge  (defined  as 
information,  facts,  and  procedures  required  for  successful  performance  in  a  domain — see 
Palumbo,  Miller,  Shalin,  &  Steel-Johnson,  2005).  There  are  at  least  four  reasons  for  doing  this. 
First,  previous  research  was  not  successful  when  it  sought  to  select  predictors  on  the  basis  of 
having  instructors  identify  individual  differences  they  perceived  to  be  relevant  to  performance 
(Schaefer,  Bencaz,  Bush,  &  Price,  2010).  This  suggested  a  new  approach  was  needed.  Second, 
measuring  prior  knowledge  is  often  an  efficient  means  of  predicting  perfonnance.  This  is  likely 
because  measuring  prior  knowledge  captures  the  joint  effects  of  domain  experience  and  general 
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mental  ability  (Borman,  White,  Pulakos,  &  Oppler,  1991;  Bonnan,  White,  &  Dorsey,  1995; 
Palumbo,  Miller,  Shalin,  &  Steel- Johnson,  2005;  Schmidt  &  Hunter,  1993;  Schmidt,  Hunter,  & 
Outerbridge,  1986).  Evidence  indicates  that  general  mental  ability  is  the  most  robustly 
predictive  of  broad  psychological  constructs  (Goska  &  Ackerman,  1996;  Gottfredson,  1998; 
Jensen,  1998;  Thorndike,  1985).  However,  general  mental  ability  would  seem  to  affect 
performance  through  the  acquisition  of  prior  knowledge.  In  addition,  experience  (often 
measured  simply  as  self-reported  length  of  time  working  in  a  given  domain)  also  affects 
performance  through  the  acquisition  of  prior  knowledge.  In  other  words,  general  mental  ability 
plus  experience  within  a  domain  contributes  to  prior  knowledge,  which  in  turn  contributes  to 
criterion  performance.  This  means  that  general  mental  ability  and  experience  significantly 
predict  prior  knowledge,  but  not  criterion  performance.  Prior  knowledge,  as  the  variable  most 
directly  related  to  criterion  performance,  does  significantly  predict  criterion  performance.  Third, 
the  most  replicated  tailored  training  effects  involve  general  mental  ability  and  prior  knowledge 
(Kalyuga,  Ayres,  Chandler,  &  Sweller,  2003;  Snow,  1991,  1992).  Fourth,  we  know  that  military 
personnel  sometimes  vary  in  amount  of  prior  knowledge  (e.g.,  with  digital  systems;  see  Bink, 
Wampler,  Goodwin,  &  Dyer,  2008). 

However,  given  the  possibility  that  other,  more  easily  acquired  measures  (e.g.,  biodata 
data  and  experience)  may  also  be  correlated  with  criterion  performance,  we  did  not  rely  solely  on 
prior  knowledge  measures.  Instead,  we  also  constructed  four  other  types  of  predictors.  First,  we 
asked  small-group  instructors  (SGIs)  in  the  Engineer  Captains  Career  Course  (ECCC)  to  predict 
the  criterion  perfonnance  of  the  officers.  Second,  we  had  the  officers  complete  a  biodata 
questionnaire  containing  general  infonnation  items  (e.g.,  military  occupational  specialty  or  MOS 
[the  officers’  career  field],  deployment  experience,  etc.).  Third,  we  constructed  experience 
scales  that  assessed  various  aspects  of  specific,  criterion-related  activities.  Fourth,  we  asked 
officers  to  rate  their  own  ability  (i.e.,  express  confidence  in  their  ability)  to  carry  out  criterion- 
related  activities. 

Our  rationale  behind  the  choice  of  these  predictor  types  was  as  follows.  First,  it  is  of 
obvious  interest  to  see  how  accurately  SGIs  can  predict  officer  performance,  as  presumably 
adjustments  to  instruction  when  made  are  often  based  on  such  judgments.  Further,  research 
indicates  that  job  supervisors  appear  to  base  their  assessments  of  supervisee  job  perfonnance 
more  on  prior  knowledge  than  actual  job  performance  (Schmidt  et  ah,  1986).  In  other  words, 
supervisor  perceptions  of  supervisee  perfonnance  are  more  conelated  with  job  knowledge  than 
with  actual  job  perfonnance.  Including  instructor  predictive  judgments  allows  us  to  see  if  a 
similar  pattern  held  for  SGI  predictions  of  officer  criterion  performance.  Second,  anecdotal 
evidence  indicates  that  many  instructors  rely  on  informal  cues  like  rank  and  deployment 
experience  to  make  predictions  about  officer  performance.  Formally  assessing  the  predictive 
power  of  such  cues  via  the  biodata  questionnaire  allowed  us  to  estimate  the  practical  utility  of 
such  information.  Third,  constructing  experience  scales  related  to  specific,  criterion-related 
activities  might  be  expected  to  yield  more  robust  prediction  than  less  targeted  predictors  like 
deployment.  Further,  the  biodata  and  experience  scales  were  intended  to  address  a  difference 
between  the  occupational  literature  and  Army  institutional  settings.  In  the  occupational 
literature,  experience  is  measured  by  simply  asking  individuals  how  long  (e.g.,  months)  they 
have  been  engaged  in  a  specific  domain  (Schmidt  &  Hunter,  1993;  Schmidt  et  ah,  1986).  Given 
the  different  duty  assigmnents  perfonned  by  U.S.  military  personnel,  analogues  of  such  simple 
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measures  (e.g.,  time-in-grade  or  time-in-service)  were  judged  unlikely  to  significantly  predict 
criterion  performance.  Fourth,  self-confidence  ratings  have  shown  significant  correlations  with 
performance  (Schaefer,  Williams,  Goodie,  &  Campbell,  2004)  and  were  therefore  worth 
including  as  a  viable  predictor  of  criterion  performance. 

Method 


Course  Selection 

Our  goal  was  to  identify  an  officer  course  with  individuals  who  varied  widely  in 
performance-relevant  experience  and  knowledge.  To  guide  our  initial  selection  of  courses,  we 
developed  seven  criteria  (Appendix  A).  We  then  began  examining  courses  listed  in  the  Army’s 
Training  Requirements  and  Resource  System  (ATRRS)  to  identify  potential  courses.  At  the 
same  time,  we  developed  interview  protocols  for  use  with  course  personnel.  The  protocols  were 
designed  to  verify  course  information  obtained  in  ATRRS  as  well  as  gather  information  on 
course  prerequisites,  officer  biodata,  and  the  nature  of  existing  course  performance  criteria. 
Interviewing  instructors  from  the  potential  course  list  as  well  as  considering  the  availability  of 
course  personnel  during  the  research  timeframe  resulted  in  the  final  selection  of  the  ECCC  at 
Fort  Leonard  Wood,  MO. 

Description  of  the  Engineer  Captains  Career  Course.  The  ECCC  is  an  officer 
professional  development  course  focused  on  training  captains  and  promotable  first  lieutenants 
for  future  duties  as  company  commanders  and  battalion/brigade  staff  officers.  Interviews  with 
course  leaders,  staff,  and  SGIs  indicated  that  officers  arrive  at  the  course  from  diverse 
backgrounds  and  widely  varied  experiences  and  knowledge.  The  course  is  2 1  weeks  long  and  is 
mixed  gender.  It  is  interesting  to  note  that  instructors  were  aware  of  subgroup  differences 
between  officers  with  prior  enlistment  experience  (i.e.,  as  noncommissioned  officers)  and 
officers  without  such  experience.  Namely,  instructors  indicated  that  prior  enlisted  officers 
tended  to  have  been  in  unifonn  much  longer  and  to  possess  a  much  more  varied  set  of 
experiences  than  non-prior  enlisted  officers.  Given  these  perceived  differences,  we  included  an 
item  in  the  biodata  questionnaire  asking  whether  or  not  the  respondent  had  prior  enlistment 
experience.  In  the  subjective  estimation  of  the  instructors,  there  is  on  average  a  50/50  split 
between  prior  and  non-prior  enlisted  officers. 

Selection  of  Performance  Criteria 

While  our  earlier  research  focused  on  the  relationship  between  broad  cognitive  traits 
(e.g.,  metacognition)  and  broad  measures  of  achievement  (e.g.,  overall  course  average — see 
Schaefer,  Bencaz,  Bush,  &  Price,  2010),  using  narrower  criteria  makes  it  easier  to  construct  good 
prior  knowledge  tests.  We  therefore  asked  instructors  about  narrower  performance  criteria 
which  exhibited  large  performance  differences.  The  instructors  indicated  that  one  such  area  was 
defensive  planning. 

Defensive  planning.  Instructors  indicated  that  many  officers  arrived  at  the  course 
without  having  practical  experience  in  any  aspect  of  military  operations  except  for 
counterinsurgency  (COIN).  Given  the  nature  of  current  military  conflicts,  many  of  the  officers’ 
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last  exposure  to  the  full  range  of  military  operations  may  have  been  in  their  Basic  Officer  Leader 
Course  (BOLC).  Instructors  further  indicated  that  the  relationships  between  similar  activities  are 
frequently  misunderstood.  For  example,  force  protection  operations  such  as  construction  of 
fortifications  or  protective  emplacements  in  a  forward  operating  base  (FOB)  and  developing 
fighting  positions  in  a  battle  position  or  company  defensive  sector  may  not  be  related  or 
sufficiently  understood  by  officers  entering  the  course.  Instructors  also  indicated  that  tactical 
fundamentals  introduced  in  defensive  planning  provided  a  foundation  and  were  built  upon  in 
later  sections  of  the  course.  From  this  point  of  view,  the  defensive  planning  exam  was  an 
important  milestone  for  course  progress. 

The  defensive  planning  exam  draws  on  basic  knowledge  of  maneuver  force  tactics, 
understanding  of  the  military  decision-making  process,  use  of  orders  and  graphics,  and  engineer 
support  of  defensive  operations  in  a  mid-  to  high-intensity  conflict.  The  defensive  planning 
exam  consists  of  two  parts.  The  first  part  is  an  objectively  scored  exam  consisting  of  18  fill-in- 
the -blank,  short  answer,  and  true/false  questions.  Possible  points  range  from  0  to  60;  ‘Go’  status 
is  achieved  by  scoring  48  or  more  points  (i.e.,  80%  or  more  correct).  The  second  part  is  graded 
on  the  basis  of  subject  matter  expertise.  While  this  task  can  be  easily  accomplished  by  course 
personnel  who  possess  the  requisite  domain  knowledge,  we  determined  that  determining  the  cues 
underlying  such  process  would  be  beyond  the  scope  of  this  project.  Thus,  we  focused  on  the  first 
part  of  the  exam. 

Participants 

Seventy-eight  (78)  ECCC  officers  and  five  (5)  SGIs  participated  in  this  research.  Of  the 
73  officers  reporting  rank  information,  72  were  Captains  and  one  was  a  First  Lieutenant 
Promotable.  All  SGIs  were  current  Captains  or  Majors.  Four  of  the  SGIs  were  in  the  U.S. 

Army,  and  one  was  a  U.S.  Marine.  Of  the  73  officers  reporting  prior  service  status,  3 1  had  prior 
enlistment  experience  and  42  did  not.  Given  the  possible  existence  of  subgroup  differences  (i.e., 
between  individuals  with  and  without  prior  enlisted  service  experience),  we  calculated  time-in¬ 
grade  and  time-in-service  (both  in  months)  for  the  groups  separately.  Time-in-grade  for  the  two 
groups  was  similar  (prior  enlisted  officers  M=  23.03,  SD  =  16.65;  non-prior  enlisted  officers  M 
=  17.93,  SD  =  14.40,  t{  1,  69)  =  1.39 ,p>  .05).  However,  the  prior  enlisted  officers  (M=  136.29, 
SD  =  61.38)  had  significantly  more  time-in-service  than  did  the  non-prior  enlisted  officers  (M= 
56.98,  SD  =  15.33,  t  (1,  69)  =  7.88,/;  <  .05). 

The  defensive  planning  exam  draws  upon  basic  knowledge  of  maneuver  force  tactics, 
understanding  of  the  military  decision-making  process,  use  of  orders  and  graphics,  and  engineer 
support  of  defensive  operations  in  a  mid-  to  high-intensity  conflict.  The  defensive  planning 
exam  consists  of  two  parts.  The  first  part  is  an  objectively  scored  exam  consisting  of  18  fill-in- 
the -blank,  short  answer,  and  true/false  questions.  Possible  points  range  from  0  to  60;  ‘Go’  status 
is  achieved  by  scoring  48  or  more  points  (i.e.,  80%  or  more  correct).  The  second  part  is  graded 
on  the  basis  of  subject  matter  expertise.  Discussion  with  the  course  instructors  indicated  that 
analyzing  the  second  part  at  the  item  level  would  require  an  amount  of  time  and  effort  which 
would  be  beyond  the  scope  of  this  project.  We  thus  focused  more  on  the  first  portion  of  the 
exam. 
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Procedure 


An  initial  group  interview  was  held  with  the  ECCC  instructors.  We  explained  that  our 
research  goal  was  to  target  a  course  criterion  which  displayed  large  differences  in  officer 
performance.  During  the  interview,  instructors  indicated  their  confidence  that  early  in  the  course 
they  could  predict  who  would  (not)  do  well  on  the  criterion.  Further  discussion  indicated  that 
trainer  assessments  were  influenced  by  general  military  bearing,  communication  skills,  and 
confidence.  After  the  defensive  planning  criterion  was  chosen,  we  developed  initial  drafts  of  the 
measures  and  submitted  them  to  ECCC  instructors  for  review.  The  instructors  agreed  that  the 
criterion  reflected  the  knowledge  and  skills  that  officers  should  retain  from  Engineer  Basic 
Officer  Leaders  Course  (BOLC)  and  related  in-unit  training  and  experiences.  It  was  reflective  of 
the  entry  level  knowledge  and  skills  for  the  ECCC.  With  this  positive  feedback,  the  instructors 
suggested  only  minor  editorial  changes.  Once  the  recommended  changes  were  made  to  the 
instruments,  all  were  approved. 

The  officer  measures  were  administered  to  the  ECCC  officers  on  the  eighth  day  of 
instruction.  Administration  took  between  one  and  two  hours  to  complete.  Participating  SGIs 
also  supplied  their  predictions  of  officer  criterion  perfonnance  on  the  eighth  day.  The  defensive 
planning  exam  took  place  on  the  twentieth  day  of  instruction.  Notably,  we  were  given  both 
digital  and  hard  copies  of  the  criterion  exam  in  advance.  This  aided  greatly  our  construction  of 
the  prior  knowledge  measure.  We  return  to  this  point  again  when  we  compare  the  skills  tapped 
by  both  measures. 

Measures 

SGI  predictions.  SGIs  were  asked  to  predict  how  officers  (either  in  their  own  group  or 
in  the  course  in  general)  would  perform  on  the  criteria.  To  make  both  the  data  collection  and 
later  statistical  analyses  tractable,  we  did  not  ask  instructors  to  rank  order  the  officers  from 
absolute  highest  to  lowest.  Instead,  we  asked  them  to  indicate  those  officers  which  they  felt 
would  fall  into  the  bottom  25%,  the  middle  50%,  or  the  top  25%  of  the  criterion  distributions 
(Appendix  B). 

Biodata  questionnaire  and  experience  and  confidence  scales.  Officers  first  read  a 
statement  of  infonned  consent  and  then  completed  a  biodata  questionnaire,  an  experience  scale, 
and  a  confidence  scale  (see  Appendix  C  for  measures  and  response  frequency  information).  We 
considered  several  factors  when  selecting  biodata  items  for  the  questionnaire.  For  example, 
instructor  interviews  indicated  that  information  such  as  MOS  for  those  with  prior  enlisted 
experience  and  deployment  infonnation  (e.g.,  location,  duty  position,  and  unit  primary  mission) 
were  used  by  instructors  to  predict  officers’  performance.  In  addition,  prior  research  found  that 
level  of  education  can  affect  predictor-criterion  relationships  (Schaefer  et  ah,  2010).  In  addition, 
it  was  found  that  amount  of  prior  military  experience  also  affected  predictor-criterion 
relationships.  This  is  relevant  because,  as  noted  above,  ECCC  instructors  knew  that  there  were 
subgroup  differences  in  their  course,  with  some  individuals  having  entered  the  Army  as  officers 
and  others  having  had  prior  enlistment  (i.e.,  non-officer)  experience. 


5 


The  following  rationale  underlay  the  use  of  experience  scales.  Schmidt,  Hunter,  and 
Outerbridge  (1986)  found  that  a  simple  index  of  experience  (e.g.,  total  months/years)  within  a 
domain  predicted  prior  knowledge.  Given  that  officers  may  fill  many  duty  positions  during  their 
service,  it  is  difficult  to  construct  such  a  direct,  simple  question  regarding  criterion-relevant 
experience.  However,  because  asking  questions  related  to  time-in-grade  and  time-in-service 
require  little  effort,  we  included  these  items  in  the  biodata  questionnaire. 

Nonetheless,  just  as  prior  knowledge  is  a  more  targeted  and  therefore  more  powerful 
predictor  of  criterion  performance  than  general  mental  ability,  perhaps  asking  targeted  (i.e., 
criterion-related)  experience  questions  would  also  prove  fruitful.  We  developed  the  experience 
scale  questions  by  tapping  engineering  skills  associated  with  defensive  planning.  The  experience 
scale  questions  asked  officers  to  indicate  whether  or  not  they  had  experience  with  various 
sources  of  civilian  training,  civilian  work  experience,  military  training,  or  military  operational 
experience  in  tasks  such  as  construction  supervision  or  gap  crossing  operations  (see  Question  12 
in  Appendix  C).  Checked  boxes  were  summed,  and  possible  responses  could  range  from  zero 
(indicating  no  relevant  training  or  experience)  to  four  (indicating  civilian  training,  civilian  work 
experience,  military  training,  and  military  operational  experience  in  the  given  task).  An  overall 
score  on  the  scale  was  computed  by  totaling  the  individual  task  scores.  The  overall  score  could 
range  from  0  to  142  (max  individual  skill  score  of  4  multiplied  by  32  items). 

Officers  were  also  asked  to  rate  their  confidence  in  their  ability  or  readiness  to  carry  out 
engineer-related  tasks.  This  was  done  by  presenting  the  officers  with  a  list  of  Engineer 
Battlefield  Functions  (e.g.,  gap  crossing  operations,  protective  emplacements)  and  asking  them 
to  use  standard  Army  training  rating  scales  (T  =  trained,  P  =  requires  training  and  practice,  and 
U=  untrained)  to  self-assess  various  competencies.  We  expected  that  confidence  might  be  based 
on  prior  knowledge,  and  hence  be  only  indirectly  related  to  criterion  perfonnance. 

Items  from  both  the  experience  and  confidence  scales  were  developed  through  a 
combination  of  rational  judgment  and  field  manual  information.  Some  of  the  items  from  the 
experience  scale  (e.g.,  plumbing,  masonry)  concern  activities  common  in  some  civilian  jobs,  but 
broadly  applicable  to  many  engineering  contexts.  Other  items  on  the  experience  and  confidence 
scales  (e.g.,  gap  crossing  operations)  are  typical  military  engineering  functions  listed  on  pages  3- 
1  and  3-2  of  Chapter  3  of  Field  Manual  3-34,  Engineer  Operations  (Department  of  the  Army, 
2009). 


Prior  knowledge  test.  The  prior  knowledge  test  (Appendix  D)  was  designed  to  assess 
the  standing  of  officers  with  regard  to  the  skills  that  the  course  builds  upon.  It  should  be  stressed 
that  this  was  a  test  of  prior  knowledge,  not  a  pretest.  An  analogy  might  help  to  clarify  our  use  of 
the  terms  prior  knowledge  and  pre-test.  Assume  that  you  are  a  Drill  Sergeant,  teaching  Basic 
Rifle  Marksmanship  to  a  group  on  new  Soldiers  in  Basic  Combat  Training.  If  you  wanted  to 
give  the  Soldiers  a  pretest  concerning  their  marksmanship  skills,  you  would  take  them  out  to  the 
range  and  have  them  go  through  the  same  marksmanship  qualification  they  will  be  seeing  at 
record  fire,  at  the  end  of  BRM.  That  is,  you  would  assess  how  the  Soldiers  currently  stand  on  the 
types  of  problems  that  you  will  be  teaching  them  how  to  handle  in  the  first  part  of  the  course.  If 
you  were  to  give  them  a  prior  knowledge  test,  however,  the  test  problems  would  assess  content 
concerning  their  general  knowledge  of  firearms,  ballistics,  and  prior  experience  with  rifle 
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marksmanship  before  entering  Basic  Combat  Training  (BCT),  such  as  hunting  or  competitive 
shooting.  This  type  of  assessment  is  looking  at  their  past  experiences  in  the  relevant  domain. 

The  pretest  would  give  you  an  idea  of  how  much  they  already  know  about  what  you  will  be 
teaching  them,  and  a  prior  knowledge  test  would  give  you  an  idea  of  how  solid  their  foundational 
skills  are — the  skills  you  will  be  expanding  and  building  upon  as  the  course  progresses. 

The  following  six  factors  drove  test  construction.  First,  military  subject  matter  expertise 
guided  the  construction  of  items  which  were  judged  to  be  either  easier  or  more  difficult  in  the 
domain  of  defensive  planning.  Second,  the  test  relied  on  officers  being  able  to  use  information 
and  apply  principles  in  the  correct  way,  not  just  to  simply  recall  or  list  facts  and  terms.  This 
served  to  highlight  differences  in  conceptual  understanding  which  might  not  be  brought  out  by 
simple  recall.  Third,  the  test  was  designed  to  provide  a  measure  of  officer  knowledge  without 
additional  resources.  Supplemental  maps,  orders,  planning  materials,  and  doctrinal  references 
were  not  required  for  successful  performance  and  were  not  provided  with  the  test.  Fourth,  the 
questions  on  the  tests  were  designed  to  prevent  easy  discrimination  between  correct  and  incorrect 
responses.  This  was  accomplished  by  including  common  errors  as  options.  Fifth,  as  noted 
above,  we  were  given  extensive  access  to  the  criterion  and  were  able  to  ensure  that  the  skills 
emphasized  on  the  criterion  were  also  being  measured  by  the  prior  knowledge  test.  Sixth,  the 
prior  knowledge  test  was  intended  to  assess  the  foundational  skills  of  incoming  officers — skills 
that  would  be  built  and  expanded  on  as  the  course  progressed.  Therefore,  we  examined  some  of 
the  skills  taught  in  the  Engineering  BOLC — a  course  that  most  if  not  all  of  the  incoming  ECCC 
officers  had  taken. 

The  criterion  measure  emphasized  eleven  essential  competencies  of  an  engineer  officer. 
To  enable  the  reader  to  gain  a  feel  for  how  the  skills  were  represented  on  both  the  prior 
knowledge  test  and  the  criterion,  in  Table  1  we  provide  a  crosswalk  of  the  tasks  and 
corresponding  skills  from  both  measures.  For  the  most  part,  the  number  of  questions  per  skill  is 
roughly  equal  across  the  two  instruments.  While  the  third  task  does  appear  to  be  an  exception, 
note  that  all  of  the  questions  on  the  prior  knowledge  exam  tapping  that  skill  are  also  interrelated 
with,  or  present  in,  other  skills.  This  illustrates  something  important  about  using  predictors  with 
this  kind  of  complex  criterion:  a  given  question  can,  and  probably  will,  tap  multiple  skills. 

The  prior  knowledge  test  measured  officer  perfonnance  on  the  eleven  skills.  It  did  so  by 
placing  the  officer  in  the  role  of  a  task  force  engineer,  who  is  planning,  supervising,  completing 
planning,  and  providing  staff  supervision  through  the  execution  of  engineer  operations 
supporting  the  defensive  mission  of  a  heavy  combined  arms  battalion  (CAB).  The  test  contained 
situational  descriptions,  tactical  diagrams  and  sketches,  graphical  symbols,  photos  of  opposing 
force  (OPFOR)  engineer  systems,  and  planning  documents. 
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Table  1 

Skill  Crosswalk  of  Prior  Knowledge  and  Criterion  Measures 


Task 

Prior 

Knowledge 

Questions 

Criterion 

Questions 

1. 

Basic  symbols,  control  measures,  and  the  tactical 
situation 

7-9,  18,20 

4-6,  13-14, 18 

2. 

Scheme  of  Obstacles  Overlay;  intent  and  effect  of 
obstacles  and  obstacle  groups 

1-7,  9-11 

1,2,4,13-14 

3. 

Directed,  reserve,  and  situational  obstacles  and  groups 

8-11,20 

4. 

Supported  force  and  engineer  organization  /task 
organization 

15-19 

12,  16 

5. 

Obstacle  and  fires  integration 

2, 5-7,9- 
10,12,14 

1,3,5,9-10,17 

6. 

Engineer  tasks  and  maneuver  commander’s  intent 

7-9,  12,14, 
16,  18-20 

1-3,6,  8,  10, 
12,15,18 

7. 

Engineer  planning  and  priorities 

3,7-11,13-19 

1,2,6,  10,  12- 
13,  15 

8. 

Capabilities  of  OPFOR  engineer  organizations, 
equipment,  and  tactics 

12,13 

7,11 

9. 

High-Value  and  High-Payoff  Targets 

12,13 

7,9 

10.  Developing  an  engagement  area  (EA) 

7 

2,5,  10 

11. 

.  Employment  of  ADAMRAAM  and  scatterable  mine 
systems 

9-11,14 

17 

The  situational  details,  diagrams,  sketches,  and  symbols  provided  were  all  items  normally 
available  to  a  task  force  engineer  from  either  the  CAB  order  or  other  military  sources.  Diagrams 
and  sketches  were  used  to  avoid  introducing  the  extraneous,  potentially  distracting  or  contentious 
details  of  maps  or  photomaps.  The  sketches  provided  unambiguous  examples  of  obstacle 
groups,  defensive  schemes,  and  possible  enemy  avenues  of  approach  to  elicit  the  officer’s 
understanding  of  the  tactical  situation.  In  this  way,  whether  an  officer  was  successful  or  not  on 
the  prior  knowledge  test  could  be  attributed  to  how  well  he  understood  (or  misunderstood)  the 
subject  matter,  and  not  to  ambiguities  in  the  test  materials. 

To  do  well  on  the  prior  knowledge  test  the  officer  had  to  meet  the  following  six 
requirements.  First,  the  officer  had  to  be  able  to  review,  analyze,  and  make  tactical  judgments 
based  on  the  provided  planning  documents.  Second,  the  officer  had  to  be  able  to  understand  and 
apply  the  engineer  doctrine  and  principles  of  defensive  planning.  For  example,  the  officer 
needed  an  understanding  of  the  integration  of  fires  and  effects  (direct  and  indirect),  maneuver, 
and  obstacles.  Third,  the  officer  had  to  be  able  to  detennine  the  effect  and  intent  of  obstacles 
from  graphics.  Fourth,  the  officer  had  to  be  able  to  specify  obstacles,  obstacle  intent,  and 
emplacement  construction  requirements  from  commander’s  intent  and  guidance.  Fifth,  the 
officer  had  to  be  able  to  understand  the  traits  of  scatterable  mine  systems  in  order  to  integrate 
their  use  into  the  overall  defense  plan.  Finally,  the  officer  had  to  understand  various  capabilities, 
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characteristics,  and  missions  of  OPFOR  engineer  units  and  systems  in  order  to  assess  target  and 
emplacement  priorities. 

Analysis  Strategy 

All  analyses  were  conducted  using  the  Statistical  Package  for  the  Social  Sciences  (SPSS 
16.0)  for  Windows,  and  the  alpha  level  for  significance  was  set  at  .05  for  all  tests.  As  this  was  an 
exploratory  analysis,  all  p  values  should  be  treated  with  caution.  We  reported  p  values  for  the 
sake  of  completeness,  but  did  not  adjust  for  family  wise  error  rate.  Any  confidence  in  the 
strength  or  pattern  of  the  relationships  should  be  tempered  in  the  absence  of  replication.  In 
analyzing  the  data,  we  used  the  following  3-stage  strategy. 

Data  screening  and  scale  construction.  First,  all  predictor  variables  were  examined  for 
problems  like  skewed  distributions  (defined  as  any  item  with  80  percent  or  more  of  responses 
falling  into  a  single  category  or  assuming  a  single  value),  truncation  of  range,  many  response 
categories  with  few  individuals,  or  insufficient  number  of  responses.  Problematic  items  were 
dropped  from  further  analysis  and  a  rationale  for  the  decision  was  given.  Second,  all  experience 
measure  items  were  grouped  into  scales  whenever  possible.  This  was  done  by  first  examining 
the  individual  question  descriptives.  If  no  problems  were  found,  then  questions  were  grouped  on 
the  basis  of  common  content  and  format.  Cronbach’s  alphas  were  then  computed  to  assess  scale 
reliability.  Unless  removing  an  item  resulted  in  an  improvement  in  the  scale’s  Cronbach’s  alpha 
by  .10  or  more  (e.g.,  the  scales  Cronbach’s  alpha  would  increase  from  .80  to  .90),  all  scales  were 
left  intact.  Potential  scale  items  without  item-level  statistical  problems  but  which  were 
insufficiently  reliable  (i.e.,  scale  reliability  was  too  low)  were  retained  as  stand-alone  predictors. 

Correlation  and  regression.  Based  on  the  Schmidt  et  al.  (1986)  findings,  we  had  four 
expectations  of  the  data.  (We  use  the  terms  ‘expectations’  and  ‘expected’  because  the  words 
‘predicted’  and  ‘predictions’  are  used  frequently  throughout  the  paper  to  refer  to  correlational 
relationships).  First,  we  expected  that  prior  knowledge  would  significantly  predict  criterion 
performance,  and  that  it  would  in  fact  be  the  strongest  predictor.  Second,  we  expected  that  one 
or  more  of  the  experience  variables  (time-in-grade,  time-in-service,  experience  scales,  and 
biodata  variables)  would  significantly  predict  prior  knowledge,  but  would  not  predict  criterion 
performance.  Third,  we  expected  that  the  SGI  predictions  would  significantly  predict  prior 
knowledge,  but  not  criterion  perfonnance.  Fourth,  we  expected  that  self-confidence  ratings 
would  significantly  predict  prior  knowledge  but  not  criterion  performance.  This  expectation  was 
derived  from  two  sources.  To  begin  with,  it  seems  plausible  that  confidence  ratings  would  arise 
(partially)  out  of  experience  in  a  domain.  If  this  is  true,  then  because  experience  is  more  directly 
related  to  prior  knowledge  than  criterion  performance,  so  too  should  this  pattern  hold  for  any 
variables  derived  from  experience.  Further,  the  judgment  and  decision  making  literature 
(Schaefer  et  al.,  2004)  indicates  that  although  significant  correlations  between  measures  of 
knowledge  and  self-confidence  ratings  are  often  obtained,  they  are  not  perfectly  correlated  and 
are  usually  biased  in  the  direction  of  overconfidence.  These  expectations,  if  met,  would  argue 
for  using  prior  knowledge  (not  experience  or  self-confidence  ratings)  to  predict  criterion 
performance.  This  is  because,  as  noted  in  the  introduction,  Schmidt  et  al.  (1986)  found  that 
experience  is  indirectly  related  to  criterion  perfonnance.  The  experience-criterion  relationship  is 
therefore  too  weak  to  serve  as  the  basis  for  making  tailored  training  decisions. 
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If  more  than  one  significant  predictor  was  found,  both  simultaneous  and  stepwise 
regressions  were  computed.  Simultaneous  regression  gives  an  estimate  of  the  upper  limits  of 
predictability,  while  stepwise  regression  estimates  the  utility  of  using  only  a  subset  of  predictors. 
This  is  useful  information,  as  combining  information  from  multiple  predictors  is  easy  when  using 
statistical  software  but  quite  burdensome  for  the  envisioned  ‘end  user’  who  is  unlikely  to  have 
access  to  this  type  of  software.  On  this  account,  we  felt  it  was  sensible  to  focus  instead  on  one  or 
two  robust  predictors  of  criterion  performance. 

Predicted  versus  observed  performance  categories.  The  third  and  final  stage  focused 
on  illustrating  how  predictor/criterion  information  could  be  translated  into  ‘user  friendly’ 
information  for  course  instructors,  managers,  and  other  relevant  personnel.  We  approached  this 
problem  in  the  following  way. 

We  followed  Cohen’s  (1992)  proposed  lower  boundary  for  a  large  effect  size  as  a 
correlation  of  .37  or  larger.  If  such  a  correlation  was  found,  we  then  subjected  the  variables  to 
both  Steps  1  and  2  (outlined  below).  If  such  a  correlation  was  not  found,  we  skipped  Step  1  and 
proceeded  to  Step  2. 

Step  1:  Total  score  relationships.  For  these  procedures,  we  visually  scanned  the 
predictor  and  criterion  total  score  frequency  distributions  to  see  if  naturally  occurring  break 
points  were  present.  To  foreshadow  our  results,  we  found  that  breaking  the  prior  knowledge  and 
criterion  distributions  into  quartiles  and  halves  was  illuminating.  (Obviously,  different  break 
points  might  be  constructed  on  the  basis  of  instructor  judgment.  For  example,  an  instructor 
might  be  interested  in  the  top  and  bottom  10  percent).  We  then  examined  the  relationship 
between  the  predictor  and  criterion  quartiles  by  constructing  a  table  indicating  the  number  of 
officers  who  were  (in)correctly  classified  on  the  basis  of  their  standing  on  the  predictor  variable. 
We  then  repeated  the  tabular  procedure,  but  this  time  compared  the  relationship  between 
predictor  and  ‘Go’  status  on  the  criterion. 

Step  2:  Subsets  of  easy  and  hard  prior  knowledge  items.  For  all  predictor/criterion  pairs 
we  attempted  to  isolate  subsets  of  the  easiest  and  hardest  prior  knowledge  items  and  assessed 
their  relationship  to  total  criterion  scores.  First,  crosstabs  between  the  easiest  and  hardest  prior 
knowledge  items  and  criterion  scores  were  constructed  to  see  if  interpretable  patterns  emerged. 
Second,  the  crosstabs  were  examined  to  see  if  there  was  any  evidence  of  an  interpretable 
relationship  between  easy/hard  item  performance  and  Go/No  Go  status  on  the  criterion. 

Results 

To  improve  readability,  we  minimized  the  presentation  of  statistics  in  the  text.  In  the  case 
of  more  complex  response  patterns,  a  verbal  summary  was  provided.  When  the  phrase  ‘most 
respondents’  was  used,  this  meant  that  more  than  80%  of  officers  gave  the  same  response,  and  by 
the  pre-defined  differential  response  rate  rule  given  above,  the  item  was  excluded  from  further 
analysis  (see  Appendix  E  for  descriptive  statistics). 
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Data  Screening  and  Scale  Construction 

Variables  were  examined  in  the  order  in  which  they  appear  in  the  Appendices  and  in 
which  they  were  described  above  (e.g.,  SGI  predictions,  biodata  questions,  experience  and 
confidence  scales,  prior  knowledge  tests,  and  criterion).  When  we  reported  criterion  statistics, 
we  did  so  both  for  total  points  and  percent  correct.  This  is  because  the  latter  makes  for  easier 
comparisons  between  various  distributions  whose  underlying  scales  may  differ  in  minimum  and 
maximum  scores.  However,  any  ‘end  user’  would  likely  use  points. 

SGI  predictions.  Although  most  SGIs  indicated  during  the  interviews  their  belief  that 
they  can  intuitively  assess  current  experience  and  knowledge  as  well  as  predict  future 
performance,  many  were  reluctant  to  make  formal  assessments  when  requested.  Further,  despite 
initial  confidence  that  accurate  intuitive  prediction  was  possible  early  on,  instructors  felt  that 
they  did  not  have  sufficient  time  with  the  officers  to  fonn  an  accurate  opinion.  The  result  for 
both  criteria  was  that  fewer  than  50%  of  all  Officers  had  SGI  predictions.  (There  were  13  SGI 
predictions  for  prior  enlisted  officers,  and  15  for  non-priori  enlisted  officers.)  However,  given  the 
possible  presence  of  subgroup  differences  we  retained  this  variable  for  further  analysis.  If 
subgroup  differences  were  present,  it  would  be  helpful  to  know  if  SGIs  perceived  the  subgroups 
differently.  One  must  remember,  however,  that  because  the  sample  was  small  we  cannot  be  sure 
of  these  correlations  until  the  research  is  replicated  with  a  larger  sample. 

Biodata  questionnaire.  All  items  in  the  biodata  questionnaire  were  examined  for 
problems.  Because  many  of  the  biodata  variables  were  dropped  in  this  stage  of  analysis,  we 
group  the  variables  into  ‘excluded’  and  ‘retained’  categories. 

Excluded  variables.  There  were  two  factors  which  caused  biodata  items  to  be  excluded 
from  further  analysis.  First,  most  respondents  provided  the  same  answers  for  rank  (almost  all 
were  Captains),  service  status  (almost  all  were  Active  Duty),  and  military  education  questions 
(almost  all  underwent  the  Advanced  Leader  Course).  These  items  were  therefore  excluded. 
Second,  there  were  many  response  categories  that  were  selected  too  infrequently.  We  therefore 
excluded  highest  rank  in  prior  service,  MOS  and  branch  in  prior  service,  and  the  schools  from 
which  any  undergraduate  degrees  were  earned. 

This  also  caused  the  deployment  variables  to  be  excluded,  replicating  a  pattern  seen  in 
our  prior  (Schaefer  et  al.,  2010)  and  current  (Schaefer,  Blankenbeckler,  &  Brogdon,  2011) 
research.  There  were  no  significant  relationships  between  the  criterion  and  dates  of  deployment, 
location  of  deployment,  duty  position,  or  primary  mission.  This  was  due  in  part  to — as  stated 
above — many  response  categories  being  too  infrequent  for  statistical  analysis.  In  other  cases, 
response  categories  were  frequent  enough  but  simply  did  not  relate  strongly  to  the  criterion. 
Multiple  attempts  were  made  to  recode  the  data  into  higher-order  categories,  but  all  such 
attempts  proved  fruitless. 

It  is  worth  noting  that  this  does  not  necessarily  preclude  this  information  from  being 
useful  for  instructors.  For  example,  prior  iterations  of  the  course  could  have  broken  along 
cleaner  lines  with,  say,  half  of  the  students  having  a  specific  deployment  experience  and  the 
other  half  not.  Such  a  pattern  would  lend  itself  both  to  instructor  perception  and  statistical 
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analysis  of  predictor/criterion  relationships.  However,  the  data  we  do  have  (combined  with  prior 
research,  as  noted  above)  does  not  engender  confidence  in  using  such  items  to  inform  tailored 
training  decisions. 

Retained  variables.  The  retained  biodata  variables  were  SGI  predictions,  time-in-grade, 
time-in-service,  prior  service  status,  level  of  civilian  education,  commissioning  source,  and 
undergraduate  major.  The  latter  two  variables  were  recoded  to  address  small  statistical  issues. 
Commissioning  source  was  recoded  to  ignore  the  ‘other’  category  as  it  contained  less  than  2 
percent  of  the  respondents.  Undergraduate  major,  as  originally  entered,  also  exhibited  the 
problem  of  too  many  categories  with  too  few  responses.  Therefore,  we  recoded  the 
undergraduate  majors  into  higher-order  categories  (e.g.,  business,  business  administration, 
entrepreneurial  studies,  etc.  were  all  recoded  as  business  degrees).  As  there  is  no  underlying 
linearity  to  the  undergraduate  major  and  commissioning  source  variables,  we  examined  their 
relationship  to  criterion  performance  via  analysis  of  variance  (ANOVA). 

Experience  scale.  The  potential  experience  scale  asked  officers  to  indicate  if  they  had 
civilian  training  or  education,  civilian  work  experience,  military  training,  or  military  operational 
experience  in  engineering  or  related  activities  like  cartography  or  obstacle  emplacement.  To 
reduce  demands  on  memory,  the  scale  simply  asked  officers  to  check  the  boxes  indicating 
whether  or  not  they  had  the  indicated  experience.  For  any  given  activity,  a  response  could  range 
from  0  (indicating  no  response  options  were  applicable)  to  4  (indicating  all  response  options  had 
been  checked).  Descriptive  analyses  of  the  questions  revealed  no  item-level  problems;  the 
Cronbach’s  alpha  of  the  scale  was  .94.  This  scale  (i.e.,  the  Defensive  Planning  Experience 
(DPE)  Scale)  was  retained. 

Confidence  scale.  The  confidence  scale  asked  officers  to  rate  their  competence  in 
carrying  out  Engineer  Battlefield  Functions  like  counter-mine  and  gap-crossing  operations. 
Descriptive  analyses  of  the  questions  revealed  no  item-level  problems;  the  Cronbach’s  alpha  of 
the  scale  was  .93.  This  scale  (i.e.,  the  Defensive  Planning  Confidence  (DPC)  Scale)  was 
retained. 

Prior  knowledge  test.  Descriptive  analysis  of  the  questions  revealed  no  item-level 
problems,  and  Cronbach’s  alpha  was  .70.  This  test  was  retained.  (See  Table  E-3,  Appendix  E, 
for  item  descriptives.) 

Criterion.  Descriptive  analysis  of  the  questions  revealed  no  item-level  problems,  and 
Cronbach’s  alpha  was  .78. 

Correlation  and  Regression 

To  ensure  clarity  of  presentation,  we  first  examined  whether  or  not  commissioning  source 
and  undergraduate  major  impacted  criterion  performance  for  either  of  the  subgroups.  Analyses 
of  variance  showed  that  undergraduate  major  did  not  significantly  impact  criterion  performance 
for  either  prior  enlisted  (F  (4,  26)  =  .10  ,p>  .05)  or  the  non-prior  enlisted  ( F  (4,  37)  =  2.55,  p  > 
.05).  Similarly,  commissioning  source  did  not  significantly  impact  criterion  perfonnance  for 
either  prior  enlisted  (F  (2,  26)  =  .34,  p  >  .05)  or  non-prior  enlisted  (F  (2,  38)  =  1 .20,  p  >  .05). 
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Next,  we  examined  whether  the  main  variables  differed  according  to  prior  enlisted 
experience.  Descriptive  statistics  from  this  analysis  are  displayed  in  Tables  2  and  3  (given  that 
level  of  civilian  education  is  not  a  truly  continuous  variable,  it  is  more  informative  to  give 
frequency  information  as  in  Table  3). 

The  subgroups  were  not  significantly  different  in  SGI  Predictions  (t  (1,  26)  =  1 .09.  p  > 
.05),  DPE  Scale  ( t  (1,  70)  =  .52,  p  >  .05),  DPC  Scale  ( t  (1,  69)  =  .47,  p  >  .05),  or  the  Prior 
Knowledge  Test  (t  (1,  70)  =  .34,  p  >  .05).  However,  the  non-prior  enlisted  officers  performed 
significantly  better  on  the  Defensive  Planning  Criterion  Exam  (t  (1,  71)  =  2.38,  p  <  .05).  In 
addition,  the  prior  enlisted  officers  possessed  significantly  more  civilian  education  (t  (1,  71)  = 
2.52,  p  <  .05)  than  the  non-prior  enlisted  officers.  This  was  due  to  the  higher  rate  of  graduate 
schooling  for  the  former  group  (see  Table  3).  Therefore,  we  correlated  the  variables  separately 
for  the  two  subgroups. 

Table  2 

Mean  Comparisons  Between  Prior  and  Non-prior  Enlisted  Officers 


Prior 

Enlisted 

Experience 

SGI 

Predictions 

DPE 

Scale 

DPC 

Scale 

Prior 

Knowledge 

Test 

(Points) 

Prior 

Knowledge 

Test 

(%  Correct) 

Criterion 

Exam 

(Points) 

Criterion 
Exam 
(%  Correct) 

Yes 

M=  2.00 
£0=  58 

M=  14.06 
£0=10.58 

M=  32.58 
SD=  8.70 

M=44.50 

£0=6.08 

M=66.42 

£0=9.07 

M=51.48 

£0=4.28 

M=85.81 

£0=7.14 

No 

M=2.21 

£O=.70 

M=  12.71 
£0=1 1.32 

M=31.60 
SD= 8.73 

M=  43.98 
£0=6.5 1 

M=  65.64 
£0=9.71 

M=53.85 

£0=4.11 

M=89.74 

£0=6.86 

Table  3 

Civilian  Education  Level  of  Prior  and  Non-prior  Enlisted  Officers 


Civilian 

Prior  Enlisted  Officers 

Civilian 

Non-prior  Enlisted  Officers 

Education 

Frequency 

Percent 

Education 

Frequency 

Percent 

Some  College 

— 

— 

Some  College 

1 

2.4 

Bachelor 

19 

61.3 

Bachelor 

34 

81.0 

Some 

Graduate 

10 

32.3 

Some 

Graduate 

7 

16.7 

Master 

2 

6.5 

Master 

— 

— 

Total 

31 

100 

Total 

42 

100 

Prior  enlisted  officers.  The  variables  in  this  analysis  were  SGI  predictions,  time-in¬ 
service,  time-in-grade,  highest  level  of  civilian  education,  Defensive  Planning  Experience  (DPE) 
Scale,  Defensive  Planning  Confidence  (DPC)  Scale,  Defensive  Planning  Prior  Knowledge  Test, 
and  the  Defensive  Planning  Criterion  Exam.  The  results  of  this  analysis  are  presented  in  Table  4. 
However,  before  proceeding  we  wish  to  iterate  once  again  the  need  for  replicating  these  findings. 
Such  caution  is  advisable  for  at  least  three  reasons.  First,  these  findings  are  based  on  a  small 
sample  of  Officers.  Second,  when  conducting  so  many  comparisons  some  will  be  statistically 
significant  as  a  matter  of  chance.  Finally,  it  is  a  truism  of  statistics  that  some  amount  of 
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2 

R"‘ shrinkage’  occurs  when  applying  a  regression  equation  based  on  one  sample  to  another 
sample. 


Table  4 

Correlations  for  Prior  Enlisted  Officers 


1 

2 

3 

4 

5 

6 

7 

8 

1. 

SGI  Predictions 

— 

.26 

.57* 

.28 

.49 

.37 

.05 

-.43 

2. 

Time-in-service 

— 

.27 

.07 

.09 

.01 

-.01 

.08 

3. 

Time-in- grade 

— 

.18 

.31 

.36* 

.00 

.09 

4. 

Civilian  Education 
Level 

— 

.22 

-.07 

-.10 

-.08 

5. 

DPE  Scale 

— 

.48* 

.10 

-.09 

6. 

DPC  Scale 

___ 

-.06 

-.08 

7. 

Prior  Knowledge 

Test 

— 

.15 

8. 

Criterion  Exam 
(Part  1) 

— 

Note :  *p  <.05.  Ns  ranged  from  13  (SGI  Prediction  correlations)  to  31. 


We  now  turn  to  our  expectations.  First,  we  expected  that  prior  knowledge  would 
significantly  predict  criterion  performance  and  that  it  would  be  the  strongest  predictor.  This 
expectation  was  not  met  as  prior  knowledge  did  not  significantly  predict  criterion  performance. 
Second,  we  expected  that  one  or  more  of  the  experience  variables  (time-in-grade,  time-in¬ 
service,  experience  scale,  and  biodata  items)  would  significantly  predict  prior  knowledge  but  not 
criterion  performance.  This  expectation  was  also  not  met.  Neither  prior  knowledge  nor  criterion 
performance  was  significantly  predicted  by  any  of  the  experience  variables.  In  fact,  it  is  quite 
striking  how  close  all  the  pertinent  correlations  are  to  zero.  Third,  we  expected  that  the  SGI 
predictions  would  significantly  predict  prior  knowledge,  but  not  criterion  performance.  This 
expectation  was  not  met,  as  the  SGI  predictions  did  not  significantly  predict  either  prior 
knowledge  or  criterion  performance.  Fourth,  we  expected  that  the  self-confidence  ratings  would 
be  significantly  correlated  with  prior  knowledge  but  not  criterion  performance.  This  expectation 
was  also  not  met.  However,  the  self-confidence  ratings  were  significantly  correlated  with  the 
experience  scale. 

Non-prior  enlisted  officers.  Again,  the  variables  were  time-in-service,  time-in-grade, 
highest  level  of  civilian  education,  DPE  Scale,  DPC  Scale,  Defensive  Planning  Prior  Knowledge 
Test,  and  the  Defensive  Planning  Criterion  Exam  (Part  1).  The  results  of  this  analysis  are 
presented  in  Table  5. 
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Table  5 

Correlations  for  Non-prior  Enlisted  Officers 


1 

2 

3 

4 

5 

6 

7 

8 

1. 

SGI  Predictions 

— 

-.05 

.04 

-.11 

.00 

.24 

.35 

.11 

2. 

Time-in-service 

— 

.81* 

.13 

.38* 

.08 

.27 

.02 

3. 

Time-in- grade 

— 

.27 

.56* 

.35* 

.34* 

.01 

4. 

Civilian  Education 
Level 

— 

.06 

-.17 

.25 

.24 

5. 

DPE  Scale 

— 

.56* 

.55* 

.16 

6. 

DPC  Scale 

— 

.22 

-.14 

7. 

Prior  Knowledge 

Test 

— 

.54* 

8. 

Criterion  Exam 
(Part  1) 

— 

Note :  *p  <.05.  TVs  ranged  from  15  (SGI  Prediction  correlations)  to  42. 


We  now  turn  to  our  expectations.  First,  we  expected  that  prior  knowledge  would 
significantly  predict  criterion  perfonnance  and,  further,  that  it  would  be  the  strongest  predictor. 
This  expectation  was  met.  Prior  knowledge  significantly  and  uniquely  predicted  criterion 
performance.  Our  second  expectation  was  also  met:  two  of  the  experience  variables  (time-in¬ 
grade  and  the  DPE  scale)  significantly  predicted  prior  knowledge,  but  did  not  predict  criterion 
performance.  Third,  we  expected  that  the  SGI  predictions  would  significantly  predict  prior 
knowledge,  but  not  criterion  performance.  This  expectation  was  not  met,  as  the  SGI  predictions 
did  not  significantly  predict  either  prior  knowledge  or  criterion  performance.  Fourth,  we 
expected  that  the  self-confidence  ratings  would  be  significantly  correlated  with  prior  knowledge 
but  not  criterion  performance.  This  expectation  was  not  met.  However,  as  with  the  prior 
enlisted  officers,  the  self-confidence  ratings  were  significantly  correlated  with  the  experience 
(DPE)  scale. 

Predicted  Versus  Observed  Performance  Categories 

As  prior  knowledge  predicted  criterion  performance  only  for  the  non-prior  enlisted 
officers,  we  conducted  this  stage  of  the  analysis  on  those  officers  only.  We  examined  both  the 
prior  knowledge  and  criterion  distributions  and  found  that  both  variables  could  be  broken  into 
quartiles  without  unduly  distorting  the  distributions  (Table  6). 
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Table  6 

Prior  Knowledge  and  Criterion  Quartiles 


Prior  Knowledge  Scores 

Criterion  Scores 

Points 

Percent  of 
Officers 

Performance 
Category  (Quartiles) 

Points 

Percent  of 
Officers 

Performance 
Category  (Quartiles) 

28-38 

23.8 

4th  Quartile  (Bottom) 

43-51 

23.8 

4th  Quartile  (Bottom) 

39-44 

26.2 

3  rd  Quartile 

52-54 

28.6 

3rd  Quartile 

46-48 

23.8 

2nd  Quartile 

55-57 

23.8 

2nd  Quartile 

49-53 

26.2 

1st  Quartile  (Top) 

58-59 

23.8 

1st  Quartile  (Top) 

Note:  Non-prior  enlisted  officers  only. 


One  way  of  understanding  the  infonnation  shown  in  Table  7  is  to  look  at  the 
categorization  errors.  The  tendency  among  this  data  set  seems  to  be  that  individuals  who  score 
in  the  bottom  half  of  the  prior  knowledge  distribution  also  score  in  the  bottom  half  of  the 
criterion  distribution.  Similarly,  individuals  who  score  in  the  top  half  of  the  prior  knowledge 
distribution  also  tend  to  score  in  the  top  half  of  the  criterion  distribution.  To  make  this  clear,  we 
collapsed  the  quartiles  into  halves  (see  Table  8). 

Table  7 

Match  Between  Prior  Knowledge  and  Criterion  Quartiles 


Prior  Knowledge 
Scores 

Actual  Criterion  Category 

Row 

Totals 

4th 

Quartile 

(Bottom) 

3rd 

Quartile 

2nd 

Quartile 

1st 

Quartile 

(Top) 

4th  Quartile  (Bottom) 

6* 

1 

3 

0 

10 

3  rd  Quartile 

3 

5* 

3 

9 

11 

2nd  Quartile 

1 

3 

2* 

4 

10 

1st  Quartile  (Top) 

0 

6 

2 

6* 

11 

Column  Totals: 

10 

12 

10 

10 

42 

Note:  Entries  in  cells  are  number  of  Captains  in  that  category.  Entries  with  an  asterisk  indicate 
correct  classifications. 
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Table  8 

Match  Between  Prior  Knowledge  and  Criterion  Halves 


Prior 

Knowledge 

Scores 

Actual 

Criterion 

Category 

Row 

Totals 

Bottom  Half 
(43-54  points) 

Top  Half 
(55-59  points) 

Bottom  Half 
(28-44  points) 

15 

6 

21 

Top  Half 
(46-53  points) 

7 

14 

21 

Column  Totals: 

22 

20 

42 

To  summarize,  those  who  scored  in  the  bottom  half  of  the  prior  knowledge  distribution 
were  more  than  twice  as  likely  to  score  in  the  bottom  half  of  the  criterion  distribution  as  in  the 
top  half.  Similarly,  those  who  scored  in  the  top  half  of  the  prior  knowledge  distribution  were 
about  twice  as  likely  to  score  in  the  top  half  of  the  criterion  distribution  as  in  the  bottom  half. 

We  next  explored  the  relationship  between  the  quartiles  and  halves  of  the  prior 
knowledge  distribution  to  ‘Go’  status  on  the  criterion  (Tables  9  and  10).  ‘Go’  status  is  defined 
by  course  personnel  as  80  percent  or  more  of  items  (i.e.,  48  or  more  points)  correct.  As  Table  9 
shows,  the  ‘No  Go’  rate  for  officers  without  prior  enlisted  experience  was  extremely  low  (9.5 
%).  The  data  show  the  same  tendency  as  in  Table  8.  Of  the  few  individuals  who  did  not  achieve 
a  ‘Go’  on  the  criterion,  three  fourths  came  from  the  bottom  half  of  the  prior  knowledge 
distribution. 

Table  9 

Match  Between  Prior  Knowledge  Quartiles  and  Criterion  Go/No  Go 


Prior  Knowledge  Scores 

Criterion  Status 

Row 

Totals 

No  Go 

Go 

4th  Quartile  (Bottom) 

2 

8 

10 

3  rd  Quartile 

1 

10 

11 

2nd  Quartile 

1 

9 

10 

1st  Quartile  (Top) 

0 

11 

11 

Column  Totals: 

4 

38 

42 

Note:  Entries  in  cells  are  number  of  Captains  in  that  category. 
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Table  10 

Match  Between  Prior  Knowledge  Halves  and  Criterion  Go/No  Go 


Prior  Knowledge  Scores 

Criterion  Status 

Row 

Totals 

No  Go 

Go 

Bottom  Half 

3 

18 

21 

Top  Half 

1 

20 

21 

Column  Totals: 

4 

38 

42 

We  then  analyzed  the  relationships  between  the  easiest/hardest  prior  knowledge  items 
and  overall  criterion  score.  (As  before,  this  was  done  only  for  the  non-prior  enlisted  officers.) 
Due  to  ties  for  item  difficulty,  we  could  not  use  the  five  easiest  items.  Therefore,  we  cross- 
tabulated  performance  on  the  seven  easiest  items  against  total  criterion  performance.  (The  seven 
easiest  items  were  Questions  1,  8b,  12f,  4b,  and  12d — see  Table  E-4,  Appendix  E.)  No 
discernible  pattern  between  easy  prior  knowledge  items  and  criterion  scores  emerged. 

Next,  we  cross-tabulated  performance  on  the  five  most  difficult  questions  (2,  17,  3d,  4e, 
and  16)  against  total  criterion  performance.  This  time,  a  meaningful  pattern  was  revealed. 
Answering  2  or  more  of  the  five  most  difficult  prior  knowledge  items  correlated  with  scoring  48 
points  or  more  on  the  criterion.  By  happenstance,  48  or  more  was  also  the  lower  bound  for  ‘Go’ 
status  (see  Table  1 1). 

Table  1 1 

Hard  Prior  Knowledge  Items  and  Criterion 


Criterion  Test 

Prior  Knowledge  Test: 

5  Hardest  Items 

0-47.5  points 

48-60  points 

0-1  Items  Correct 

4 

21 

2  or  More  Items  Correct 

0 

17 

Discussion 

It  is  obvious  that  the  relationships  among  the  variables  differed  markedly  between  the 
prior  and  non-prior  enlisted  officers.  To  gain  a  better  understanding  of  the  nature  of  these 
differences  we  first  recap  how  our  expectations  were  met  (or  not)  by  the  data  from  the  two 
subgroups.  Our  first  expectation  was  that  prior  knowledge  would  significantly  predict  criterion 
performance  and  do  so  more  powerfully  than  any  other  included  predictor.  This  was  not  true  for 
the  prior  enlisted  officers,  but  was  true  for  the  non-prior  enlisted  officers.  Our  second 
expectation  was  that  one  or  more  of  the  experience  variables  would  significantly  predict  prior 
knowledge  but  not  criterion  performance.  This  was  not  true  for  the  prior  enlisted  officers,  but 
was  true  for  the  non-prior  enlisted  officers.  Our  third  expectation  was  that  the  SGI  predictions 
would  significantly  predict  prior  knowledge,  but  not  criterion  performance.  This  expectation 
was  not  met  for  either  subgroup.  Our  fourth  expectation  was  that  self-confidence  ratings  would 
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significantly  predict  prior  knowledge,  but  not  criterion  perfonnance.  This  was  not  met  for  either 
subgroup.  However,  what  was  true  of  both  subgroups  is  that  the  self-confidence  ratings 
significantly  predicted  the  experience  (DPE)  scale  ratings. 

Considered  as  a  whole,  the  most  important  finding  is  the  presence  of  systematic 
relationships  among  experience,  prior  knowledge,  and  criterion  performance  only  for  those 
without  prior  enlisted  experience.  One  possible  interpretation  of  this  finding  is  that  the  measures 
behaved  differently  for  the  two  groups.  For  example,  perhaps  the  prior  knowledge  scores  or 
scales  exhibited  different  ranges,  variance,  or  factorial  structures  of  the  prior  knowledge  tests  (as 
revealed  by  item-difficulties)  between  the  subgroups.  This  was  not  the  case,  however.  One 
striking  feature  of  these  results  is  the  similarity  between  the  two  subgroups  on  prior  knowledge 
scores  and  the  scales  of  experience  and  confidence.  On  this  basis,  the  expectation  that  these 
subgroup  differences  are  due  to  markedly  different  ranges  or  variances  was  not  tenable.  Nor  is 
there  evidence  that  the  factorial  structure  of  the  prior  knowledge  test  was  markedly  different 
between  the  two  subgroups,  as  the  correlation  between  prior  knowledge  item  difficulties  for  the 
two  subgroups  was  .89  (N=67,p  <  .001).  The  obtained  subgroup  differences  cannot  therefore  be 
attributed  to  either  a  statistical  artifact  or  structural  bias  in  the  prior  knowledge  test. 

How  then  should  we  understand  these  group  differences?  In  considering  this  question,  it 
is  useful  to  graphically  compare  the  findings  of  Schmidt  et  al.  (1986)  to  our  non-prior  enlisted 
officers.  A  more  complete  comparison  is  enabled  by  considering  the  contribution  of  supervisor 
ratings  in  Schmidt  et  al.  which  found  the  ratings  to  be  significantly  correlated  with  prior 
experience,  but  not  criterion  (work  sample)  performance.  This  seems  to  suggest  that  while  using 
experience  to  predict  criterion  performance  is  not  the  best  choice  for  predicting  performance,  it  is 
not  necessarily  an  irrational  choice.  There  are  systematic  relationships  among  these  variables. 
The  problem  is  that  experience  is  more  indirectly  related  to  criterion-relevant  knowledge  than  is 
a  test  of  prior  knowledge. 

Does  this  same  pattern  hold  for  the  non-prior  enlisted  officers?  Recall  that  SGI 
predictions,  as  we  argued  earlier,  are  a  reasonable  analog  to  the  supervisor  ratings  from  Schmidt 
et  al.  The  independent  and  theoretically  grounded  evidence  for  this  relationship  established  by 
Schmidt  et  al.  provides  sufficient  reason  to  justify  revisiting  this  relationship.  As  Figure  1 
shows,  the  similarities  between  the  Schmidt  et  al.  data  and  that  of  our  non-prior  enlisted  officers 
is  striking,  with  SGI  predictions  maintaining  a  substantial  relationship  with  prior  knowledge  but 
a  weak  relation  with  the  criterion,  just  as  found  by  Schmidt  et  al.  This  global  pattern  of  relations 
again  suggests  that  experience  is  a  rational,  if  suboptimal,  predictor  of  criterion  perfonnance. 
Schmidt  et  al.  interpreted  this  to  mean  that  prior  knowledge  is  more  accessible  than  direct 
criterion  performance  to  supervisors.  In  other  words,  through  interacting  with  a  supervisee  the 
supervisor  may  be  able  to  assess  whether  or  not  the  supervisee  can  ‘talk  the  talk’  but  not  whether 
they  can  ‘walk  the  walk.’  To  the  extent  that  ‘talking’  correlates  with  ‘walking’,  the  supervisor 
ratings  will  be  valid  predictors  of  criterion  perfonnance. 
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Schmidt,  Hunter,  &  Outerbridge  (1986) 


Non-prior  Enlisted  Officers 


Figure  1.  Comparison  of  Schmidt,  Hunter,  and  Outerbridge  (1986)  data  (N=  1,474)  with  non¬ 
prior  enlisted  officer  data. 

Note.*p  <  .05. 

The  parallels  displayed  in  Figure  1  are  even  more  compelling  when  one  considers  the 
methodological  differences  between  the  Schmidt  et  al.  analysis  and  the  current  effort.  First,  the 
instruments  and  methods  used  in  this  report  and  those  of  Schmidt  et  al.  are  not  the  same. 
Consider  the  differences  between  ‘Work  Sample’  and  Defensive  Planning  Part  1  (i.e.,  the 
objectively  scored  portion)  criterion  performance.  A  ‘work  sample’  is  a  demonstration  of 
proficiency  in  a  work-related  task.  In  contrast,  Part  1  criterion  performance  was  an  academic 
test.  Second,  the  supervisor  ratings  were  summary  scores  derived  from  ratings  on  14  job 
performance  dimensions.  The  SGIs,  in  contrast,  were  merely  asked  to  predict  criterion 
performance.  Third,  it  seems  likely  that  the  supervisors  had  much  more  time  to  assess  worker 
skill  than  the  SGIs  did  to  assess  officer  skill.  This  is  because  the  average  job  tenure  for 
supervisees  in  the  Schmidt  et  al.  research  was  approximately  two  years  as  compared  to  the  eight 
days  that  the  SGIs  had  to  observe  their  officers  in  this  research.  Fourth,  the  work  samples  from 
Schmidt  et  al.  are  described  as  being  simulations  of  important  job-related  tasks.  This  suggests 
that  each  work  sample  was  a  relatively  simple  task  (although  many  different  skills  might  be 
tapped  across  multiple  work  samples).  In  contrast,  the  defensive  planning  exam  involved 
multiple  skills. 

The  different  ways  of  eliciting  ‘supervisor  ratings’  and  ‘SGI  ratings’  are  also  important. 
Supervisor  ratings  are  usually  elicited  via  a  Likert  scale,  but  the  SGI  ratings  here  placed  less 
demand  upon  instructors  by  simply  asking  them  to  place  individuals  in  the  bottom  quartile, 
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middle  50  percent,  or  top  quartile  of  the  criterion  distribution.  Further,  the  manner  in  which 
experience  was  measured  was  also  quite  different.  Schmidt  et  al.  examined  studies  which  simply 
asked  individuals  how  long  (e.g.,  years  and  months)  they  had  been  involved  in  a  given  job 
domain.  In  this  research,  we  asked  specific  questions  about  criterion  related  activities. 

Having  established  strong  parallels  between  the  results  for  the  non-prior  enlisted  officers 
and  the  Schmidt  et  al.  data  even  in  the  face  of  substantial  methodological  differences,  we  are 
now  in  a  position  to  better  understand  our  obtained  subgroup  differences.  Figure  2  brings  into 
sharper  focus  the  systematic  nature  of  the  subgroup  differences.  Here  we  focus  on  the  different 
ways  in  which  the  SGI  ratings  function  between  the  subgroups.  Namely,  for  non-prior  enlisted 
officers,  the  SGI  ratings  are  reasonably  correlated  with  prior  knowledge  and  weakly,  but 
positively,  related  to  criterion  performance.  This  was  not  true  for  the  prior  enlisted  officers. 
There,  the  SGI  ratings  were  not  significantly  correlated  with  prior  knowledge,  and  negatively 
and  substantially  (albeit  nonsignificantly,  given  the  small  number  of  SGI  ratings)  correlated  with 
criterion  performance. 


Non-prior  Enlisted  Officers  Prior  Enlisted  Officers 


Figure  2.  Comparison  of  non-prior  enlisted  officer  data  with  prior  enlisted  officer  data. 
Note*  p  <  .05. 


What  might  be  causing  these  subgroup  differences?  Given  the  correlational  nature  of  our 
data  any  posited  reasons  must  remain  speculative.  We  are  more  confident  that  the  findings  for 
the  non-prior  enlisted  officers  would  replicate,  given  that  they  themselves  appear  to  be  a 
replication  of  the  model  found  in  Schmidt  et  al.,  than  we  are  that  the  pattern  found  for  the  prior 
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enlisted  officers  would  do  so.  On  the  face  of  it,  there  seem  to  be  at  least  four  plausible 
hypotheses  regarding  this  data  set. 

First,  as  might  be  expected  given  the  differences  in  time-in-service  noted  earlier,  the  prior 
enlisted  officers  tended  to  be  older.  Perhaps  the  classroom  environment  required  them  to  build 
upon  their  existing  prior  knowledge  in  a  way  that  tapped  study  skills  and  habits  that  were 
stronger  in  the  younger,  more  recent  college  graduates  (i.e.,  the  non-prior  enlisted  officers). 

These  differences  in  study  skills  need  not  themselves  be  correlated  with  having  certain 
experiences  that  increase  engineering-related  knowledge,  but  might  instead  become  salient  only 
in  a  classroom  environment.  Second,  motivational  factors  could  be  at  play.  The  prior  enlisted 
officers  have  already  (in  some  cases)  had  relatively  lengthy  careers  in  the  military,  and  may  not 
be  looking  to  ‘make  their  mark’  in  quite  the  same  way  as  the  younger,  non-prior  enlisted  officers. 
Their  expectations  for  future  promotions  or  desire  to  compete  for  coveted  assignments  may  be 
lower  than  their  younger  peers.  Third,  the  instructors  hypothesized  that  some  of  the  prior 
enlisted  officers  might  be  ‘recycles’ — that  is,  this  may  not  have  been  their  first  attempt  at  passing 
the  ECCC.  If  this  is  so,  then  it  might  be  that  those  individuals  possessed  enough  domain 
familiarity  to  do  well  on  the  prior  knowledge  test,  but  not  enough  so  that  their  knowledge 
provided  a  sufficiently  firm  foundation  for  expanding  upon  the  skills  they  already  possessed. 
Fourth,  recall  the  fact  that  what  SGI  ratings  we  do  have  appear  to  be  either  uncorrelated  (in  the 
case  of  prior  knowledge)  or  even  negatively  correlated  (in  the  case  of  criterion  perfonnance) 
with  relevant  prior  enlisted  officer  variables.  This  suggests  that  the  two  officer  populations  may 
have  been  treated  differently  during  training.  Again,  however,  these  are  raised  as  merely 
plausible  hypotheses.  Verifying  one  or  more  of  these  would  require  replications  with  another 
sample  of  ECCC  officers  using  additional  measures  and  perhaps  (in  the  case  of  the  last 
hypothesis)  with  a  researcher  observing  instruction. 

Recommendations 

The  fact  that  the  non-prior  enlisted  officer  data  so  closely  mirrors  meta-analytic  findings 
based  on  much  larger  sample  sizes  gives  us  confidence  in  the  reliability  of  that  data  set.  Further, 
the  fact  that  the  two  subgroups  were  different  on  not  only  instruments  devised  by  our  research 
team,  but  also  in  how  the  SGI  predictions  relate  to  prior  knowledge  and  criterion  performance 
argues  against  attributing  these  subgroup  differences  to  sampling  error.  These  recommendations 
overlap  considerably  with  those  in  the  companion  report  on  predicting  noncommissioned  officer 
(NCO)  course  perfonnance  (Schaefer,  Blankenbeckler,  &  Brogdon,  2011)  as  there  are 
similarities  in  the  findings.  The  recommendations  are  given  in  the  subheadings  which  follow. 
Recall  that  we  are  interested  in  predictor  measures  as  they  help  instructors  to  determine  which 
individuals  require  tailored  training. 

Use  Prior  Knowledge  as  a  Predictor 

When  possible,  using  prior  knowledge  as  a  predictor  is  a  good  bet.  As  discussed  in  the 
introduction  of  this  paper,  prior  knowledge  captures  the  joint  effects  of  both  mental  ability  and 
experience  within  a  domain.  This  was  borne  out  by  the  fact  that  prior  knowledge  alone 
significantly  predicted  performance. 
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Focus  on  Narrow  Criteria  to  Maximize  Utility  of  Predictive  Information 

In  our  prior  research  (Schaefer  et  ah,  2010)  we  focused  on  broad  psychological  traits  and 
criteria  (e.g.,  metacognition  and  class  averages).  However,  given  the  relative  success  of  using 
prior  knowledge  measures  as  predictors  a  different  tack  is  advisable. 

Constructing  prior  knowledge  measures  that  attempt  to  draw  on  the  content  of  an  entire 
course  seem  ill  advised.  First,  developing  and  administering  such  a  measure  would  take  an 
inordinate  amount  of  time.  Second,  it  is  unclear  how  useful  such  infonnation  would  be.  If  an 
individual  perfonns  poorly  on  all  portions  of  the  test,  would  the  instructor  (even  if  willing)  be 
able  to  tailor  the  entire  course  around  that  person?  Third,  such  an  approach  does  not  lend  itself  to 
measurement  throughout  a  course. 

It  seems  more  feasible  to  use  ‘mini’  prior  knowledge  tests  prior  to  blocks  of  instruction  or 
training  on  tasks  that  are  important  in  terms  of  cost,  core  objectives,  foundational  knowledge  and 
skills,  or  difficulty  level.  Then  decisions  can  be  made  regarding  what  kind  of  training  (if  any)  is 
warranted  on  that  particular  block  of  training. 

Use  Biodata  Variables  Judiciously 

The  general  types  of  biodata  variables  which  instructors  might  use  to  assess  current  and 
future  performance  were  not  predictive.  This  might  be  because  of  statistical  issues  (arguably,  the 
failure  of  the  deployment  variables  to  predict  might  be  because  there  were  too  few  categories 
into  which  responses  fell)  or  because  the  variables  were  only  indirectly  related  to  criterion 
performance  (as  was  the  case  for  the  experience  scale,  and  even  more  so  for  the  confidence 
scale). 


However,  it  seems  that  using  biodata  variables  to  identify  subgroup  differences  is 
promising.  This  report  and  our  prior  research  (Schaefer  et  al.,  2010)  identified  at  least  two  Army 
courses  in  which  subgroups  exhibited  starkly  different  predictor-criterion  relationships.  It  is 
encouraging  that  in  both  of  these  courses  the  subgroups  (consisting  of  differences  in  military 
experience)  were  brought  to  our  attention  by  the  course  instructors,  indicating  that  such 
differences  are  sometimes  suspected  by  course  personnel. 

Estimate  Total  Score  and  Easy/Hard  Item  Relationships  When  Validating  Prior 
Knowledge  Predictors 

If  the  correlation  between  prior  knowledge  and  criterion  total  score  is  large  enough  (using 
our  given  rule,  .37  or  more)  then  cross-tabulations  can  be  used  to  generate  infonnation  usable  by 
course  personnel.  Such  information  can  then  be  leveraged  to  categorize,  on  the  basis  of  observed 
probabilities,  the  likely  future  criterion  perfonnance  throughout  the  entire  examined  criterion 
range.  Further  infonnation  can  be  gleaned  by  examining  the  ability  of  hard  (and,  in  theory,  easy) 
prior  knowledge  items  to  predict  criterion  performance.  However,  it  is  important  to  realize  that 
even  a  relatively  strong  correlation  of  .50  or  greater  might  not  reveal  itself  in  the  first  cross¬ 
tabulation  that  is  constructed.  It  might  take  several  different  breakouts  of  the  data  (e.g.,  into 
thirds,  fourths,  or  fifths)  before  a  clear  pattern  emerges. 
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When  no  large  (.37  or  greater)  total  prior  knowledge-criterion  score  correlation  is  present 
it  is  still  possible  to  use  sets  of  difficult  items  to  predict  who  will  do  extremely  well.  As  the 
current  data  demonstrated,  even  when  a  strong  correlation  is  present,  using  subsets  of  difficult 
prior  knowledge  items  can  be  helpful.  It  is  important  to  realize  that  the  payoff  of  using  difficult 
items  is  also  heavily  dependent  on  the  failure  rate  on  the  criterion.  The  utility  of  this  approach 
would  be  much  more  evident  if  the  criterion  failure  rate  was  larger. 

Explore  the  Predictor-Criterion  Relationship  in  Multiple  Ways 

Determining  how  to  examine  the  relationship  between  predictor  and  criterion  involves 
the  simultaneous  consideration  of  the  strength  and  nature  of  the  predictor/criterion  relationship, 
instructor  perceptions  about  what  performance  is  acceptable  and  what  is  not  (these  perceptions 
may  or  may  not  be  the  same  as  pre-established  ‘Go/No  Go’  standards),  and  what  type  of  tailored 
training  is  desired  (remedial,  mastery,  or  both). 

The  predictor/criterion  relationship.  If  there  is  an  unusually  strong  and  reliable 
correlation  between  the  predictor  and  criterion  (.70  or  higher,  say)  then  there  are  several  options 
open  to  course  personnel.  First,  relationships  between  the  predictor  and  criterion  distributions 
should  be  explored  using  crosstabs  procedures  (see  Tables  7  through  10).  Where  to  make  the 
cuts  can  be  detennined  by  a  variety  of  factors.  For  example,  simple  scanning  of  the  frequency 
tables  may  indicate  naturally  occurring  breakpoints.  However,  cross-tabulating  the  resulting 
categories  may  not  reveal  a  relationship  even  if  a  strong  overall  correlation  is  present.  Given  the 
strong  correlation,  we  know  that  such  a  relationship  exists.  Therefore,  it  is  up  to  the  individual  to 
find  the  right  cut  points.  (For  example,  we  knew  that  a  strong  correlation  existed  between  the 
prior  knowledge  test  and  the  criterion  exam.  When  we  broke  the  prior  knowledge  and  criterion 
measures  into  thirds,  however,  that  relationship  was  largely  obscured.)  How  to  choose  the 
correct  cut  points  will  involve  some  trial  and  error.  The  cut  points  might  be  determined  by  what 
constitutes  Go/No  Go  on  the  criterion  or  by  other  naturally  occurring  break  points  in  the 
predictor  and/or  criterion  distributions. 

Whether  or  not  a  strong  correlation  exists,  it  also  useful  to  explore  how  well  easy  and 
difficult  prior  knowledge  items  predict  both  criterion  points  and  Go/No  Go  rates.  The  former  is 
a  bit  trickier  to  determine,  as  the  relationship  between  criterion  points  and  difficult  items  might 
also  require  some  trial  and  error.  For  example,  consider  Table  11.  It  was  just  happenstance  that 
the  relationship  between  difficult  prior  knowledge  items  and  criterion  points  coincided  with  the 
Go/No  Go  boundary.  When  mapping  prior  knowledge  easy/difficult  item  performance  onto 
Go/No  Go  rates,  the  process  a  little  easier  as  only  the  prior  knowledge  item  dimension  can  vary. 

Instructor  perceptions  of  acceptable  performance.  These  perceptions,  as  noted  above, 
may  vary  from  Go/No  Go  rates.  For  example,  perhaps  the  instructor  wishes  to  really  hone  the 
skills  of  his  officers.  In  that  case,  the  instructors  internal  perception  of  acceptable  behavior  may 
exceed  the  ‘Go’  rate  official  standard. 

The  type  of  tailored  training  that  is  desired.  This  is  not  truly  independent  of  the 
preceding  subsection.  If  tailoring  for  remedial  training  alone  is  the  goal,  then  the  individual  will 
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be  most  interested  in  the  relationships  between  the  predictor  and  the  bottom  end  of  the  criterion 
distribution.  This  will  also  probably  mean  that — if  subsets  of  prior  knowledge  items  are  used — 
the  focus  will  be  largely  on  easy  items,  as  low  perfonning  individuals  will  be  the  ones  most 
likely  to  fail  such  items.  Conversely,  if  the  goal  is  mastery  training,  then  the  focus  will  be  on  the 
relationship  between  the  predictor  and  the  upper  end  of  the  criterion  distribution.  If  subsets  of 
prior  knowledge  items  are  used  in  that  case,  the  focus  will  be  on  the  difficult  items. 

Trade  offs  in  categorization.  It  is  easy  to  misunderstand  that  there  are  tradeoffs 
involved  in  categorization.  If  the  goal  is  to  make  absolutely  sure  that  only  those  who  truly  need 
remedial  training  are  the  ones  receiving  it,  then  the  risk  is  that  individuals  who  might  have 
benefited  from  remedial  training  are  not  receiving  it.  Say  for  example  that  an  instructor  finds 
that  individuals  who  score  in  the  bottom  25%  group  on  a  prior  knowledge  measure  often  end  up 
in  the  bottom  10%  group  on  the  criterion  distribution.  To  ensure  that  only  the  truly  needy 
receive  remedial  training  the  instructor  has  to  decide  that  only  individuals  who  score  in  the 
bottom  10%  of  the  prior  knowledge  distribution  will  receive  remedial  training.  In  that  case, 
individuals  who  score  between  the  10th  and  25th  percentiles  on  the  prior  knowledge  test  might 
benefit  from  remedial  training,  but  fail  to  receive  it. 

Conversely,  an  instructor  might  be  truly  interested  in  mastery  training  only  for 
individuals  who  show  extreme  skill  on  the  criterion.  Say  further  that  the  instructor  has  found 
that  individuals  who  score  in  the  top  25%  group  on  the  prior  knowledge  measure  end  up  in  the 
top  10%  on  the  criterion  measure.  The  instructor  realizes  that  individuals  who  score  in  the  top 
25%  on  the  prior  knowledge  measure  might  be  able  to  even  further  improve  their  perfonnance 
by  being  given  advanced  training  (e.g.,  more  complicated  and  demanding  materials,  more 
practice,  etc).  However,  the  instructor  wants  to  make  sure  that  only  the  individuals  in  the 
uppermost  top  of  the  prior  knowledge  distribution  will  receive  such  training.  In  this  way,  the 
instructor  decides  that  only  individuals  who  score  in  the  top  10%  of  the  prior  knowledge 
distribution  will  receive  such  training.  In  the  mirror  image  of  the  above  risk  analysis,  now 
individuals  scoring  between  the  90th  and  75th  percentiles  on  the  prior  knowledge  distribution 
might  benefit  from  such  mastery  training,  but  fail  to  receive  it. 

How  large  such  tradeoffs  could  be  will  depend  on  the  specifics  of  the  data  set  and  the 
purpose  of  the  course.  But  such  tradeoffs  should  be  kept  in  mind  when  detennining  how  to 
leverage  predictor  information  in  making  tailored  training  decisions. 

Conclusions 

In  sum,  making  intelligent  tailored  training  decisions  based  on  individual  differences  is 
challenging  and  will  require  a  unique  blend  of  testing  and  subject  matter  expertise.  The  need  for 
testing  expertise  is  obvious,  requiring  knowledge  of  test  construction  and  validation  procedures. 
However,  the  need  for  subject  matter  expertise  is  at  least  as  (if  not  more)  important.  Subject 
matter  experts  will  be  required  to  help  test  creators  detennine  suitable  items  for  tapping 
prerequisite  skill,  knowledge,  and  experiences,  and  to  help  test  creators  craft  predictor  measures 
that  address  the  instructor’s  needs  (e.g.,  identify  individuals  who  will  need  assistance,  identify 
individuals  who  should  be  challenged  or  can  assist  others).  In  addition,  subject  matter  experts 
can  help  test  creators  determine  what  kinds  of  biodatas  should  be  included  to  test  for  relevant 
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subpopulation  differences.  The  subpopulation  differences  found  in  our  prior  (Schaefer  et  ah, 
2010)  and  present  research  were  brought  to  our  attention  by  course  personnel  prior  to  test 
construction.  Developing  research  teams  with  the  appropriate  psychometric  and  military 
expertise  will  require  careful  investment  of  resources,  further  suggesting  the  need  for  targeting 
areas  in  which  tailoring  will  yield  the  most  benefit. 
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Appendix  A 

Course  Selection  Criteria 


1 .  Number  of  officers  in  each  course  -  Each  officer  arrives  at  a  course  with  his/her  own  KSE, 

gained  over  years.  Therefore,  theoretically,  the  larger  the  number  of  officers  in  a  course, 
the  greater  the  “potential”  for  differences  in  KSE.  However,  keep  in  mind  that  even 
though  there  might  be  a  large  number  of  officers,  it’s  possible  that  a  majority  will  have 
similar  KSE,  with  only  some  minority  having  different  KSE.  Ensure  that  selected  courses 
have  a  large  enough  “sample  size”  of  officers  with  differing  KSE. 

Guideline:  Courses  with  a  larger  number  of  officers  are  more  likely  to  have  more  differences  in 
KSE. 

2.  Multiple  MOSs  -  Each  MOS  (and  branch/specialty  for  officers)  of  the  Army  has  some  unique 

training  requirements,  skills  and  tasks.  Therefore,  personnel  from  varied  MOSs 
(branches/specialties)  will  arrive  at  a  course  with  differing  KSE.  However,  keep  in  mind 
that  even  though  there  might  be  a  large  number  MOSs  (branches/specialties),  it’s  possible 
that  a  majority  of  officers  will  have  a  common  MOS  (branch/specialty),  with  only  some 
minority  being  a  different  MOS  (branch/specialty).  Ensure  that  selected  courses  have  a 
large  enough  “sample  size”  of  officers  with  different  MOS  (branch/specialty). 

Guideline:  The  larger  the  variety  of  MOSs  (and  branch/specialty  for  officers)  attending  the 

course,  the  greater  the  likelihood  of  differences  in  KSE.  Also  consider  that  some  MOSs 
(branch/specialties)  are  so  different  that  those  attending  a  course  will  increase  the 
likelihood  of  different  KSE.  (Example:  Soldiers  from  infantry,  armor  and  even  engineer 
areas  are  much  more  similar  in  many  aspects  of  KSE  than  Soldiers  from  chaplain 
assistant  or  transportation  areas.)  An  ideal  situation  would  be  a  course  with  2-3  well- 
represented,  qualitatively  different  MOSs. 

3.  Course  Length  (topic/subject)  -  With  the  exception  of  Initial  Entry  Training  (IET)  courses, 

longer  courses  (more  than  45  days)  are  generally  for  NCO  and  officer  professional 
development  and  are  not  usually  focused  on  a  specific  skill  or  capability.  As  the  level  of 
the  course  increases  (e.g.  from  ALC  [E-6]  to  SLC  [E-7]  or  from  Officer  Basic  Courses 
[0-1]  to  Captains’  Career  Courses  [0-3])  the  military  KSE  will  likely  increase. 

Personnel  attending  the  higher  level  courses  will  have  had  more  time-in-service  and  more 
assignments.  However,  the  overall  general,  military  experience  will  become  more 
common  as  the  time-in-service  increases.  Keep  in  mind  that  the  focus  is  on  the  technical 
skill  areas  (not  soft  skills)  which  will  only  be  a  portion  of  the  course. 

Guideline:  Generally,  the  shorter  courses  that  are  not  designed  for  a  specific  MOS/branch  are 
more  likely  to  have  differences  in  more  general  KSE,  while  the  longer  professional 
development  courses  will  have  greater  differences  in  specific  military  assignment  KSE 
areas.  Consider  only  technical  portions  of  professional  development  courses. 
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4.  Course  Content  -  The  nature  of  the  course  content  (“soft  skills”  versus  technical  skills)  will 

have  implications  for  how  easily  prior  knowledge  can  be  measured  or  how  easily 
performance  can  be  measured.  Generally,  need  to  consider  the  technical  task  areas  for 
courses  where  prior  knowledge  can  be  measured  and  avoid  attempts  to  measure  “soft 
skill”  areas.  Consider  blocks  of  training  within  courses  rather  than  an  entire  course, 
especially  if  the  block  of  training  is  a  critical  technical  skill  area.  Also,  officers  are  more 
likely  to  possess  differences  in  KSE  in  the  more  technical  areas  than  in  the  “soft  skill” 
areas. 

Guideline:  Differences  in  KSE  will  generally  be  more  important  in  courses  and  blocks  of  training 
with  structured,  sequential  technical  skill  areas  that  are  critical  for  course  completion. 
Unstructured  and  non-sequential  courses  and  blocks  of  training  will  generally  involve 
more  “soft  skill”  areas  and  the  differences  in  KSE  will  have  less  impact. 

5.  Prerequisites  -  Officers  attending  higher  level  courses  (e.g.,  Sergeant  Major  Academy  as 

opposed  to  SLC  or  ALC)  will  generally  begin  the  course  with  a  more  common  skill  level 
in  the  area  to  be  trained  in  the  course.  If  course  prerequisites  are  established  and 
enforced,  the  likelihood  of  prior  KSE  that  could  impact  the  course  training  may  be 
minimal. 

Guideline:  “Basic”  and  “intermediate”  level  courses  are  more  likely  than  more  “advanced”  level 
courses  to  have  officers  with  differences  in  KSE  that  matter. 

6.  Mandatory  course  completion  -  Courses  that  must  be  successfully  completed  to  continue 

Service  within  the  military  (e.g.,  professional  development  courses  versus  basic  digital 
skills)  are  more  likely  to  have  officers  attending  with  greater  differences  in  KSE.  The 
intent  of  the  courses  is  generally  to  allow  officers  to  “cross-level”  the  military 
experiences  they  have  gained  so  all  can  move  forward  with  a  more  common  and 
complete  understanding  of  the  military. 

Guideline:  Mandatory  professional  development  courses  are  more  likely  to  have  measurable 
differences  in  KSE  than  more  general  subject  area  courses.  Consider  only  technical 
portions  of  professional  development  courses,  not  the  general  “soft  skills”. 

7.  Volunteer  or  selected  for  course  -  Generally,  courses  with  attendees  who  must  volunteer  (e.g., 

Airborne)  are  generally  people  who  perceive  a  beneficial  outcome  from  the  completion  of 
the  course,  either  personal  gratitude  or  professional  enhancement.  Personnel  who  are 
selected  for  course  attendance  based  on  some  criteria  (e.g.,  Drill  Sergeant)  may  not  have 
the  same  perceptions  or  motivation.  Selection  criteria  will  usually  consider  identifiable 
areas  of  KSE.  Therefore,  it  could  be  presumed  that  courses  with  all  volunteers  are  more 
likely  to  have  a  greater  difference  in  KSE  than  courses  with  central  selection  processes. 

Guideline:  Courses  that  have  both  volunteers  and  selectees  have  a  high  possibility  of  extreme 
differences  of  KSE,  as  well  as  all  volunteer  courses. 
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Other  Considerations 


1 .  Number  of  courses  that  can  be  affected  -  Once  potential  courses  for  differences  in 
KSE  have  been  identified,  one  of  the  down-select  factors  should  consider  the  number  of  similar 
courses  taught  at  multiple  locations  who  could  benefit  from  the  results  of  this  investigation;  to 
provide  the  Army  a  “bigger  bang  for  the  buck.” 

2.  Decisions  as  to  which  courses  to  examine  for  this  project  can  be  based  on  the 
established  criteria.  In  this  decision  process,  interactions  between/among  criteria  should  also  be 
considered  as  an  important  factor.  Since  only  5  courses  will  be  selected  to  visit  to  gather 
information  on  potential  KSE  to  measure,  a  further  consideration  is  the  number  of  potential 
courses  at  an  installation  that  offer  potential.  That  is,  if  multiple  courses  offer  the  same  potential 
for  measuring  KSE,  priority  should  be  given  to  multiple  courses  at  the  same  installation  in  order 
to  maximize  benefit  of  travel. 

At  the  end  of  this  criteria  definition  process,  we  will  compile  the  assessment  for  each 
criterion  for  10  courses  (some  information  will  come  from  web  sites  and  other  from  telephone 
calls).  When  pertinent  information  is  available  we  will  establish  a  relatively  simple  check  list  to 
apply  to  the  courses  (see  below).  Keep  in  mind,  our  purpose  in  this  exercise  is  to  identify  the  5 
courses  we  would  like  to  visit  to  help  determine  which  KSE  and  what  measures  would  be  most 
appropriate.  Something  like  the  following  rating  scale  might  work. 

Use  a  rating  scale: 


0  3  5 

(Very  slim  chance  of  differing  KSE)  (Almost  certain  of  differing  KSE) 


Selection  Criteria 

Course 

1 

2 

3 

4 

5 

6 

7 
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Small  Group  Instructor  Predictions 

The  purpose  of  this  form  is  to  gain  insight  to  your  intuition  and  observations  in  assessing  officer 
knowledge,  skills,  and  experiences.  Many  trainers  have  indicated  that  they  are  able  to  assess 
officer  potential  and  performance  in  general  and/or  for  specific  subjects  and  skills  early  in  the 
course.  Please  rate  the  officers  in  your  instructional  group  and  any  other  officers  in  the  course 
that  your  intuition,  observations,  or  impressions  have  caused  you  to  assess.  Place  an  X  or  S  in 
the  appropriate  box  for  Tactics  and  Defensive  Operations. 


Place  an  X  or  ^  in  the  appropriate  boxes  for  your  assessment. 


Officer 

Roster 

Number 

Assessment  of  the  Officers  Future  Academic 
Performance 

Performance  on  the  Defensive  Module  Planning 
Examination 

Top  25% 

Middle  50% 

Lower  25% 

Cannot 

Evaluate 

Officers  in  My  Group 

Other  Officers  in 
the  Course 
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Biodata  Questionnaire  and  Experience  Scales 

BIOGRAPHICAL  INFORMATION 

1.  Class  Number 


2.  Rank  (circle  one) 

1LT  1  LT  (P)  CPT  Other 


TIG/TIS 

Years 

Months 

3.  Time-in-grade 

4.  Time-in-service 

5.  Source  of  Commission  (circle  one) 

ROTC  USMA  OCS  Other 


6.  Did  you  have  prior  enlisted 
or  warrant  service? 

YES  NO  (If  yes,  see  7.A) 

6.A.  Highest  Rank: 

MOS: 

Service/Branch: 

7.  Service  Status  (circle  one) 

Active  Duty  National  Guard  US  Army  Reserve 

8.  Military  Education  Level  (circle  all  that  apply) 

BNCOC/ALC  ANCOC/SLC  WOC  BOLC 
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9.  Civilian  Education  Level  (circle  highest  level  of  education) 

Non  HSG  GED  HS  Diploma  Some  College  (no  degree) 

Associates  Degree  Bachelors  Degree  Graduate  Work  Master’s  Degree 


9.  A.  If  undergraduate  degree,  state  the  degree/major/school: 


9.B.  If  graduate  degree,  state  the  degree/major/school: 


10.  Deployment  History  (Most  recent  first) 

Date:  From 

Date:  To 

Iraq 

Afghan 

Other 

Duty  Position 

Unit  Primary 
Mission 

e.g.  Jun, 
2007 

Jul,  2008 

AT 

Platoon  Leader 

Route  Clearance 

(Continue  if  more  deployment  experience) 
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11,  Assignment  History  (Most  recent  first) 

Date:  From 

Date:  To 

Battalion 

Brigade 

Division 

Duty  Position 

e.g.  Feb, 
2009 

Apr,  2010 

EGR 

CO/BSTB 

4BCT 

82  Abn  Div 

XO 

Feb,  2007 

Jan,  2009 

EGR 

CO/BSTB 

4BCT 

82  Abn  Div 

PL 

(If  prior  service,  provide  3  years  prior  to  commissioning.) 
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12.  Individual  skills  training  and  experience  (check  all  that  apply) 


Skill  or  Expertise 

Source(s)  of  Education,  Training,  and  Experience 

Civilian 

Training  and/or 
Education 

Civilian 

Work 

Experience 

Military 

Training 

Military 

Operational 

Experience 

carpentry,  roofing,  &  framing 

plumbing 

masonry 

paving,  road  building,  and  repair 

construction  equipment  operation 

construction  supervision 

precision  survey 

cartography 

photogrammetry 

imagery  interpretation 

terrain  analysis 

soil  analysis 

map  production 

water  purification 

water  distribution 

waste  disposal 

physical  security 

countermobility  planning 

obstacle  construction  & 
emplacement 

lethal 

non- lethal 

mobility  planning 

mobility  operations 

obstacle  breaching  &  reduction 

counter-IED  operations 

gap  crossing  operations 

bridging  and  river  crossing  ops 

fighting  &  protective  emplacements 

camouflage  &  concealment 

deception  operations 

damage  assessment 

damage  control 

preparing  construction  materials 
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13.  Knowledge  and  Skills  Proficiency.  Provide  a  self-evaluation  of  your  competency  to 
execute  the  following  engineer  battlefield  functions.  (Check  the  most  appropriate  answer.) 
Trained  =  I  could  successfullv  plan  and  supervise  execution  of  this  function. 

Require  Training  &  Practice  =  I  would  be  capable  of  correctlv  performing  most  planning 
and  execution  aspects  of  this  function. 

Untrained  =  I  require  additional  training  to  be  able  to  correctlv  perform  the  planning  and 
execution  aspects  of  this  function. 

Engineer  Battlefield  Function 

Trained 

Require 
Training  & 
Practice 

Untrained 

Mobility 

Counter-mine/IED/  obstacle  operations 

Gap  crossing  operations 

River  crossing  operations 

Construction/clearing  of  roads  and  trails 

Forward  aviation  combat  engineering 

Countermobility 

Mine  operations 

Obstacle  development 

Survivability 

Emplacements  and  fighting  positions 

Protective  emplacements 

Protected  support  facilities 

Camouflage 

Concealment 

Deception 

General  Engineering 

Line  of  Communication  (LOC)  construction/repair 

Logistics  -  support  facilities  construction 

Area  damage  control 

Construction  -  materials  production 

Civic  Action  Projects 

Security  assistance  training  and  assistance  teams 

Topographic  Engineering 

Terrain  analysis 

Precision  survey 

Map  production 
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Appendix  D 

Defensive  Planning  Prior  Knowledge  Test 

Percent  of  individuals  who  correctly  answered  a  question  is  located  below  each  item  stem?. 
Non-prior  enlisted  denoted  by  NPE,  prior  enlisted  denoted  by  PE. 

General  Instructions:  These  questions  will  not  be  used  for  academic  evaluations  in  Engineer 
Captains  Career  Course.  They  will  only  be  used  to  assess  your  knowledge  and  skills  on  selected 
subjects  as  you  arrive.  If  you  are  uncertain  of  the  correct  answer,  leave  it  blank.  Record  only 
those  answers  that  you  believe  are  correct. 

GENERAL  SITUATION:  The  Peoples  Republic  of  Canto  (PRC),  without  provocation  attacked  Blount 
and  seized  control  of  the  Soto  Region.  In  response,  the  United  States  deployed  JTF  Kilo,  consisting  of  II 
Corps,  the  9th  Infantry  Division  (MECH)  (-),  the  16th  Infantry  Division  (AASLT),  and  sustainment  and 
support  units  as  part  of  a  NATO  and  regional  coalition.  Coordinated  attacks  of  II  Corps  supported  by 
coalition  air  have  virtually  destroyed  the  PRC’s  7th  Armored  Division  and  defeated  the  14th  Infantry 
Division  (Heavy)  and  27th  Armored  Brigade  (Sep).  The  4th  HBCT,  9th  ID  has  been  in  pursuit  of  fleeing 
remnants  of  14th  Infantry  Division.  However,  intelligence  sources  and  reports  from  the  4th  HBCT’s  ISR 
Squadron,  4/88th  Cav,  indicate  that  additional  PRC  forces  have  massed  at  the  border  and  fresh  enemy 
reconnaissance  forces  have  crossed  into  Blount.  4th  HBCT  has  ordered  3-19  INF  BN  (CAB)  to  establish 
an  area  defense  and  block  PRC  counterattacks  while  the  remainder  of  4th  HBCT  and  the  9th  ID  move 
forward.  The  3- 19th  INF  BN  (CAB)  has  been  reinforced  with  additional  engineer  assets  from  the  377th 
Maneuver  Enhancement  Brigade. 

SPECIAL  SITUATION:  You  have  just  arrived  in  the  theater  of  operations  and  have  been 
assigned  to  4th  HBCT,  9th  ID.  Upon  reporting  to  the  HBCT  Main  Command  Post  (CP),  you  were 
told  that  the  previous  TF  Engineer  for  3-19  INF  (CAB),  has  been  injured  and  evacuated.  You 
are  to  immediately  take  his  place.  Upon  arrival  at  the  3-1 9th  INF  BN’s  Main  CP,  you  find  the 
staff  completing  planning  and  maneuver  units  are  occupying  initial  positions.  The  XO  and  S3, 
both  pleased  to  see  you,  tell  you  to  review  the  TF  Engineer’s  notes  and  plans  and  prepare  to 
coordinate  defensive  preparations. 

You  find  early  planning  sketches  that  correspond  to  the  defensive  course  of  action  graphics. 
Engagement  Area  (EA)  Brown  was  identified  in  initial  planning  as  critical  to  the  Battalion’s 
defense.  Answer  questions  1-2  referring  to  the  sketch  below: 
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1 .  What  was  the  intent  of  the  obstacle  group  designed  for  EA  Brown?  (Circle  the  correct 
answer.) 


[Question  1:  NPE  57.1%  correct,  PE  70  %  correct] 

A.  Slow  the  enemy  attack  to  pennit  the  defender  time  to  acquire,  target,  and  destroy  enemy 
vehicles  and  formations  and/or  delay  the  enemy  force  to  permit  the  friendly  force  to  break 
contact  and  disengage.  (Fix  Effect) 

B.  Break  up  enemy  formations  and  tempo,  allowing  some  elements  of  the  enemy  force  to  bypass 
obstacles  while  other  elements  deploy  early  and  breach.  (Disrupt  Effect) 

C.  Divert  the  enemy  from  an  avenue  of  approach  and  allow  or  force  their  formations  bypass  into 
a  desired  direction  or  a  prepared  engagement  area.  (Turn  Effect) 

D.  Create  a  situation  in  which  massed  fires  and  obstacles  halt  the  attack  along  an  avenue  of 
approach  or  prevent  the  enemy  from  passing  through  the  engagement  area.  (Block  Effect) 

E.  All  the  above  are  supported  by  the  obstacle  group  in  EA  Brown. 
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2.  This  obstacle  group  should  be  integrated  with  the  effects  of  direct  and  indirect  fires  and 
enhance  these  effects.  What  characteristics  of  defensive  fires  and  effects  should  the  obstacle 
group  in  EA  Brown  enhance?  (Circle  the  correct  answer.) 

[Question  2:  NPE  33.3  %  correct,  PE  60%  correct] 

A.  The  massing  of  direct  and  indirect  fires  across  the  entire  enemy  avenue  of  approach  to  halt 
the  enemy  advance  and  attrite  his  forces. 

B.  The  massing  of  fires  into  restrictive  terrain  or  anchor  points  for  obstacles  to  prevent  bypass  or 
breach  of  obstacles. 

C.  The  impact  of  interlocking  fires  and  fires  from  varied  positions  into  channelized  enemy 
formations,  forcing  them  to  fight  in  multiple  directions  simultaneously. 

D.  None  of  the  above  would  be  characteristics  of  fires  in  this  EA. 

Disruptive  obstacle  groups  were  considered  in  the  forward  area  of  the  defense.  The  obstacle 
group  sketch  below  is  an  example  of  initial  planning  in  the  CAB’s  security  area.  Answer 
questions  3-4  referring  to  the  sketch  below  and  your  knowledge  of  disruptive  obstacles  in  the 
defense. 
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3.  From  the  characteristics  and  planning  considerations  below,  select  those  which  are  valid  for 
planning  obstacles  and  obstacle  groups  to  facilitate  the  disruptive  effect.  (Place  an  “X”  in  the 
blanks  for  all  that  apply;  one  or  more  responses  are  correct.) 

[Question  3a:  NPE  73.8  %  correct,  PE  60.0%  correct] 

[Question  3b:  NPE  40.5  %  correct,  PE  23.3%  correct] 

[Question  3c:  NPE  50.0  %  correct,  PE  26.7%  correct] 

[Question  3d:  NPE  23.8  %  correct,  PE  33.3%  correct] 

_ A.  The  obstacle(s)  should  attack  (influence)  approximately  half  of  the  expected  enemy 

avenue  of  approach. 

_ B.  Obstacles  should  be  more  easily  detected  as  the  enemy  nears  them. 

_ C.  Initial  obstacles  should  appear  more  complex  than  those  in  the  desired  direction  of 

enemy  movement. 

_ D.  Obstacles  should  require  less  extensive  resources  (labor,  time,  equipment,  materials, 

etc.). 
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4.  What  is  the  desired  effect  of  disruptive  obstacles  in  the  security  area?  (Circle  all  that  apply; 
one  or  more  responses  are  correct.) 

[Question  4a:  NPE  57.1  %  correct,  PE  53.3%  correct] 

[Question  4b:  NPE  97.6  %  correct,  PE  96.7%  correct] 

[Question  4c:  NPE  73.8  %  correct,  PE  83.3%  correct] 

[Question  4d:  NPE  50.0  %  correct,  PE  56.7%  correct] 

[Question  4e:  NPE  9.5  %  correct,  PE  16.7%  correct] 

A.  Divert  the  enemy  off  his  intended  avenue  or  approach  or  attack  routes  and  on  to  the  avenues 
that  best  support  our  scheme  of  maneuver  and  his  destruction. 

B.  Halt  the  enemy  advance. 

C.  Break  up  the  tempo  of  the  attack  by  forcing  some  enemy  elements  to  deploy  and  breach 
early. 

D.  Slow  the  attack  to  permit  time  for  targeting  and  destruction  of  enemy  forces  or  friendly  force 
disengagement  and  repositioning. 

E.  Deceive  the  enemy  as  to  the  exact  locations  of  our  defenses. 

F.  Delay  some  elements  of  the  attacking  force,  disrupting  command  and  control  of  the  attack 
and  causing  piecemeal  commitment  of  enemy  forces. 
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The  example  below  depicts  varied  obstacle  groups  along  enemy  avenues  of  approach  through  a 
company  position.  Normally,  a  company-team  will  have  the  mission  to  cover  only  one  or  two 
obstacle  groups  in  the  defense.  Answer  questions  5-6  referring  to  the  sketch  below: 


5.  Match  the  Obstacle  Groups  with  the  desired  effect  that  the  commander  desires  along  each 
enemy  avenue  of  approach.  (Enter  the  letter  for  the  obstacle  effect  beside  the  Obstacle  Group. 
Obstacle  effects  may  be  used  more  than  once  or  not  at  all.) 

[Question  5-1:  NPE  45.2  %  correct,  PE  33.3%  correct] 

[Question  5-2:  NPE  66.7  %  correct,  PE  66.7%  correct] 

[Question  5-3:  NPE  92.9  %  correct,  PE  90.0%  correct] 


A.  Disrupt  Effect 

Obstacle  Group  1 

B.  Obstruct  Effect 

Obstacle  Group  2 

C.  Turn  Effect 

Obstacle  Group  3 

D.  Block  Effect 

E.  Fix  Effect 
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6.  Along  which  Enemy  AA  would  you  expect  to  find  the  greatest  concentration  of  planned 
massed  direct  and  indirect  fires  integrated  with  the  obstacles?  (Select  one  answer.) 

[Question  6:  NPE  42.9  %  correct,  PE  50.0%  correct] 

_ A.  Avenue  of  Approach  1 

_ B.  Avenue  of  Approach  2 

_ C.  Avenue  of  Approach  3 

_ D.  Planned  direct  and  indirect  fires  would  be  equally  distributed  across  all  AAs. 
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The  selected  Course  of  Action  for  the  CAB  defense  is  indicated  in  the  next  sketch.  The  BN 
CDR  has  provided  the  following  guidance  for  the  conduct  of  the  defense  and  the  engineer 
obstacle  effort: 


“I  want  to  stop  and  destroy  the  enemy  in  EA  Blue.  However,  we  must  initially  slow  his 
advance  as  he  enters  our  sector  and  deceive  him  as  to  the  position  and  strength  of  our 
defenses.  Be  sure  that  we  can  get  C  Co.  out  of  their  initial  positions  and  back  into  the 
depth  defenses.  As  the  enemy  enters  our  defenses,  force  him  into  our  kill  zones  and  prevent 
his  use  of  other  approaches.  Finally,  as  he  enters  EA  Blue  slow  his  movement  rate  and 
attrite  him  heavily,  then  hold  him  while  we  finish  the  fight  and  complete  the  destruction  of 
his  forces.” 
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7.  The  BN  CDR  has  approved  the  priorities  for  the  obstacle  groups  indicated  by  the  green 
numbers  on  the  yellow  polygons.  Indication  of  the  desired  obstacle  effect  should: 

•  Drive  integration  of  obstacles  and  fires 

•  Focus  subordinate  and  supporting  staff  fire  planning 

•  Focus  the  obstacle  effort 

•  Multiply  the  effects  of  firepower 

Based  on  the  commander’s  guidance  for  obstacles  and  priorities,  what  obstacle  effects  symbol 
should  be  associated  with  the  obstacle  groups  and  priority  numbers  indicated  in  the  situation 
overlay  above?  (Match  the  letter  of  the  effects  symbol  to  the  obstacle  group  priority  number. 
Symbols  may  be  used  more  than  once  or  not  at  all.) 

[Question  7-1:  NPE  76.2  %  correct,  PE  76.7  %  correct] 

[Question  7-2:  NPE  45.2  %  correct,  PE  53.3  %  correct] 

[Question  7-3:  NPE  64.3  %  correct,  PE  73.3  %  correct] 

[Question  7-4:  NPE  59.5  %  correct,  PE  53.3  %  correct] 


Obstacle  Group 
Priority 

Effects 

Letter 

Effect  Symbol 

1. 

2. 

3. 

4. 

A 

B 

*  #  •* 

**  »».«* 

C 

D 

«-V- 

E 

F 
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As  defensive  preparations  have  progressed,  you  have  tracked  obstacle  completion.  Obstacle 
groups  are  emplaced  as  indicated.  Scatterable  minefields  are  planned  and  approved  as  situational 
or  reserve  obstacles.  Lane  Alice  is  has  been  created  to  facilitate  rapid  repositioning  C  Company. 
Refer  to  the  defensive  sketch  below  when  answering  questions  8  -  20. 


D-l  1 


8.  Lane  Alice  has  been  constructed  to  provide  a  double  lane  vehicular  route  through  the  obstacle 
group  in  the  eastern  part  of  EA  BLUE.  From  the  list  below  select  the  characteristics  that  should 
define  LANE  ALICE.  (Circle  all  that  apply;  one  or  more  responses  are  correct.) 


[Question  8a:  NPE  57.1  %  correct,  PE  60.0  %  correct] 

[Question  8b:  NPE  100.0  %  correct,  PE  96.7  %  correct] 

[Question  8c:  NPE  50.0  %  correct,  PE  63.3  %  correct] 

[Question  8d:  NPE  38.1  %  correct,  PE  36.7  %  correct] 

[Question  8e:  NPE  52.4  %  correct,  PE  50.0  %  correct] 

[Question  8f:  NPE  69.0  %  correct,  PE  73.3  %  correct] 

[Question  8g:  NPE  73.8  %  correct,  PE  80.0  %  correct] 

A.  Lane  Alice  will  be  a  clear  route  through  all  obstacles. 

B.  Lane  Alice  should  be  at  least  (1)  one  meter  wide  with  tracing  tape  down  the  center. 

C.  Lane  Alice  should  be  at  least  (8)  eight  meters  wide. 

D.  Lane  Alice  should  be  at  least  (15)  fifteen  meters  wide. 

E.  Lane  Alice  should  be  straight  and  follows  a  proscribed  azimuth  making  marking  optional 
along  the  lane. 

F.  Specific  responsibilities  will  be  identified  for  closure  of  Lane  Alice  and  execution  of  the 
associated  reserve  targets. 

G.  Lane  Alice  should  include  sudden  turns  or  “traps”  to  prevent  enemy  exploitation  of  the  route. 
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9.  The  CAB  Commander  has  planned  several  scatterable  minefield  obstacles  as  situational  or 
reserve  obstacles  using  area-denial  artillery  munitions  and  remote  antiannor  mines  [ADAMS- 
RAAM],  modular  pack  mine  system  [MOPMS],  and  Volcano.  All  minefields  have  been 
approved  by  the  9th  ID.  Based  on  your  review  of  the  situation,  how  are  scatterable  minefield 
obstacles  in  or  near  EA  BLUE  and  EA  WHITE  being  employed?  (Circle  the  correct  response.) 

[Question  9:  NPE  50.0  %  correct,  PE  56.7  %  correct] 

A.  To  separate  attacking  enemy  echelons. 

B.  To  shape  the  battle  space  for  the  deep  battle. 

C.  To  defeat  or  repair  expected  breach  or  by-pass  efforts  or  close  a  lane. 

D.  To  emplace  additional  obstacles  that  are  production  shortfalls  (supporting  engineers  were  not 
able  to  accomplish  due  to  priorities,  time,  materials,  or  equipment). 
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10.  The  Volcano  minefields  south  southeast  of  EA  WHITE  and  west  of  EA  BLUE  are 
situational  obstacles.  Given  the  commander’s  intent  and  priorities  for  these  obstacle  groups, 
what  are  some  of  the  basic  principles  for  planning,  preparing,  and  executing  situational  obstacles 
that  should  be  followed?  (Circle  all  that  apply;  one  or  more  responses  are  correct.) 

[Question  10a:  NPE  50.0  %  correct,  PE  76.7  %  correct] 

[Question  10b:  NPE  85.7  %  correct,  PE  90.0  %  correct] 

[Question  10c:  NPE  38.1  %  correct,  PE  46.7  %  correct] 

[Question  lOd:  NPE  64.3  %  correct,  PE  56.7  %  correct] 

[Question  lOe:  NPE  88.1  %  correct,  PE  76.7  %  correct] 

A.  The  obstacles  should  be  fully  integrated  with  friendly  direct  and  indirect  fires  and  effects. 

B.  Specific  friendly  or  enemy  situation  triggers  (criteria)  should  be  established  for  employment 
of  these  targets. 

C.  A  Volcano  launch  system  should  be  identified  and  munitions  dedicated  for  both  targets. 

D.  Both  targets  should  be  executable  simultaneously  without  degrading  the  effects  of  the  other. 

E.  Volcano  should  only  be  used  for  situational  targets  since  they  are  always  time  sensitive  or 
short-notice  requirements. 
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1 1 .  MOPMS  have  been  planned  to  defeat  or  repair  expected  enemy  breach  or  by-pass  efforts, 
close  lanes,  and  reinforce  existing  obstacles.  Identify  characteristics  of  MOPMS  that  makes 
them  ideal  for  these  missions.  (Circle  all  that  apply;  one  or  more  responses  are  correct.) 

[Question  11a:  NPE  66.7  %  correct,  PE  66.7  %  correct] 

[Question  lib:  NPE  71.4  %  correct,  PE  56.7  %  correct] 

[Question  11c:  NPE  85.7  %  correct,  PE  83.3  %  correct] 

[Question  lid:  NPE  47.6  %  correct,  PE  60.0  %  correct] 

[Question  lie:  NPE  81.0  %  correct,  PE  73.3  %  correct] 

A.  Small  size  and  weight  make  MOPMS  ideal  for  backpacking  long  distances. 

B.  Mines  have  a  standard  4-day  (96-hour)  lethal  duration  after  employment. 

C.  Only  when  triggered  by  a  direct  wire  link  can  MOPMS  self-destruct  (SD)  times  be  recycled 
and  dispersed  mines  command-detonated. 

D.  Using  the  M7 1  remote-control  unit,  an  operator  can  control  up  to  15  MOPMS  or  groups  out 
to  a  range  of  1,000  meters. 

E.  Mines  can  be  recovered  and  reloaded  for  use  if  not  detonated. 
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12.  Based  on  the  terrain,  the  friendly  defense  plan,  and  the  obstacle/barrier  plan,  which  enemy 
combat  engineer  systems  shown  below  should  you  recommend  for  consideration  as  high-payoff 
targets  (HPTs)  for  the  CAB  defense  both  north  and  south  of  Oak  Creek?  (Circle  all  that  apply; 
one  or  more  responses  are  correct.) 


[Question  12a:  NPE  78.6  %  correct,  PE  93.3  %  correct] 
[Question  12b:  NPE  83.3  %  correct,  PE  90.0  %  correct] 
[Question  12c:  NPE  47.6  %  correct,  PE  63.3  %  correct] 
[Question  12d:  NPE  97.6  %  correct,  PE  100.0  %  correct] 
[Question  12e:  NPE  59.5  %  correct,  PE  80.0  %  correct] 
[Question  12f:  NPE  100.0  %  correct,  PE  96.7  %  correct] 
[Question  12g:  NPE  95.2  %  correct,  PE  96.7  %  correct] 
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A.  Armored  mine  clearing  launcher  systems 


B.  Tanks  fitted  with  mine  rollers  and  plows 


C.  Tank  launched  bridges 


D.  Dump  trucks 


E.  Armored  route  clearing  tractors 


F.  Road  graders 


G.  None  of  the  above.  In  this  situation  HPTs  should  be  limited  to  Tanks  and  Infantry  Fighting 
Vehicles  (IF Vs). 
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13.  During  offensive  operations,  how  would  the  enemy  employ  armored  mine  laying  systems 
like  the  GMZ  pictured  below?  (Circle  all  that  apply;  one  or  more  responses  are  correct.) 


[Question  13a:  NPE  64.3  %  correct,  PE  56.7  %  correct] 
[Question  13b:  NPE  69.0  %  correct,  PE  66.7  %  correct] 
[Question  13c:  NPE  85.7  %  correct,  PE  73.3  %  correct] 
[Question  13d:  NPE  38.1  %  correct,  PE  46.7  %  correct] 


A.  Hold  them  in  reserve  until  his  forces  assume  the  defense  or  are  ordered  to  defend. 

B.  As  an  element  of  a  mobile  obstacle  detachment,  emplace  hasty  mine  fields  along  a  vulnerable 
flank  to  block  blue  force  counter  attacks. 

C.  Employ  them  in  their  secondary  role  as  infantry  carriers  for  support  troops  or  to  haul  barrier 
materials  or  supplies. 

D.  Employ  them  as  an  element  of  the  antitank  reserve  to  block  and  destroy  blue  force 
counterattack  forces  that  penetrate  the  attacking  fonnations  or  threaten  supply  routes. 
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14.  Two  ADAMS-RAAM  obstacles  have  been  integrated  into  the  maneuver,  fires,  and  obstacle 
plans.  Additional  munitions  have  been  made  available  for  these  targets.  What  are  some  of  the 
considerations  when  planning  ADAMS-RAAM  obstacles?  (Circle  all  that  apply;  one  or  more 
responses  are  correct.) 

[Question  14a:  NPE  81.0  %  correct,  PE  66.7  %  correct] 

[Question  14b:  NPE  57.1  %  correct,  PE  50.0  %  correct] 

[Question  14c:  NPE  97.6  %  correct,  PE  83.3  %  correct] 

[Question  14d:  NPE  45.2  %  correct,  PE  56.7  %  correct] 

A.  These  targets  will  be  fired  by  organic  systems  of  the  CAB  and  will  compete  for  other 
requests  for  indirect  fires  and  expend  critical  munitions. 

B.  These  munitions  will  be  fired  by  a  combination  of  organic  CAB  assets  and  supporting  field 
artillery  units  based  on  the  scheme  of  indirect  fires  and  the  availability  of  delivery  systems. 

C.  Triggering  the  target  and  delivery  of  planned  ADAMS-RAAM  minefields  is  the  sole 
responsibility  of  the  Task  Force  Engineer. 

D.  Once  approved,  the  Fire  Support  Element  in  coordination  with  the  assigned  Field  Artillery 
delivery  unit  will  be  responsible  to  determine  the  technical  aspects  of  delivery  -  range-to-target, 
time,  rounds  per  aim  point,  number  of  aim  points,  etc. 

15.  Available  engineer  (sapper)  platoon  hours  have  been  used  emplacing  minefields  and  wire 
obstacles  and  some  organic  engineer  excavation  resources  (ACEs  and  HMEEs)  have  been  used 
constructing  the  three  antitank  ditches.  Given  the  modularity  of  the  HBCT  and  the  CABs,  how 
many  vehicle  fighting  positions  (to  protect  Mis  and  M2s)  would  be  required  for  the  CAB 
companies  fighting  pure  in  the  defense?  (Circle  the  correct  answer.) 

[Question  15:  NPE  64.3  %  correct,  PE  76.7  %  correct] 


A. 

9 

C. 

14 

B. 

12 

D. 

19 
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16.  Uninterrupted  delivery  of  fires  and  smoke  are  critical  to  the  plan.  The  3- 19th  INF  (CAB) 
commander  has  directed  that  while  protection  of  direct  fire  systems  is  critical,  protecting  the 
CAB’s  mortar  platoon  is  absolutely  essential.  He  wants  both  primary  and  alternate  positions 
prepared  and  all  of  their  organic  vehicles  to  be  hull  down.  How  many  hull  down  positions  must 
be  added  to  your  survivability  plan?  (Circle  the  correct  answer.) 

[Question  16:  NPE  0.0  %  correct,  PE  10.0  %  correct] 

A.  6  C.  10 

B.  8  D.  12 


17.  Five  (5)  D7  bull  dozers  have  been  attached  for  the  mission.  What  would  be  the  most  likely 
source  (or  sources)  for  these  and  other  excavation  resources?  (Circle  the  correct  answer.) 

[Question  17:  NPE  31.0  %  correct,  PE  33.3  %  correct] 

A.  The  heavy  equipment  platoon  of  the  4th  HBCT’s  organic  combat  engineer  battalion 

B.  The  heavy  construction  company,  organic  to  the  combat  engineer  battalion  of  the  9th  ID 
(MECH). 

C.  An  engineer  battalion  assigned  to  the  maneuver  enhancement  brigade  supporting  the  4th 
HBCT,  9th  ID,  or  II  Corps  and  the  JTF. 

D.  The  general  headquarters  (GHQ)  supporting  II  Corps  and  the  JTF 
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18.  Identify  factors  that  you,  as  the  TF  Engineer,  should  consider  when  developing  the  work 
plan  and  obstacle  execution  matrix  for  the  available  engineers  and  excavation  resources?  (Circle 
all  that  apply;  one  or  more  responses  are  correct.) 

[Question  18a:  NPE  95.2  %  correct,  PE  96.7  %  correct] 

[Question  18b:  NPE  90.5  %  correct,  PE  86.7  %  correct] 

[Question  18c:  NPE  83.3  %  correct,  PE  93.3  %  correct] 

[Question  18d:  NPE  42.9  %  correct,  PE  33.3  %  correct] 

[Question  18e:  NPE  81.0  %  correct,  PE  90.0  %  correct] 

[Question  18f:  NPE  71.4  %  correct,  PE  63.3  %  correct] 

A.  Time  available  to  prepare  the  defense 

B.  Construction  materials  or  munitions  required  and  available 

C.  Priorities  established  by  the  maneuver  commander 

D.  Fuel  consumption  rates  for  all  excavation  systems,  troop  carriers,  and  prime  movers. 

E.  Excavation  or  emplacement  systems  available  and  their  projected  operational  readiness 
rate(s) 

F.  Travel  time  between  primary  work  sites 

19.  The  S2  has  indicated  that  the  enemy  will  be  capable  of  attacking  six  (6)  hours,  sooner  than 
anticipated.  There  is  not  sufficient  time  to  excavate  all  survivability  positions  planned.  In 
addition  to  informing  the  CAB  Commander  and  S3,  what  actions  should  you  take?  (Circle  the 
correct  answer.) 

[Question  19:  NPE  52.4  %  correct,  PE  60.0  %  correct] 

A.  Continue  working  the  priorities  established  by  the  CAB  Commander. 

B.  Change  to  the  priorities  requested  by  the  company/  team  commanders. 

C.  Give  priority  to  the  thinner  skinned  M2s. 

D.  Give  priority  to  the  faster  firing  Mis. 

E.  Change  all  remaining  positions  to  hull  defilade  positions  to  speed  the  process. 
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20.  Lane  Alice  is  the  route  for  movement  of  C  Company  to  the  alternate  position  in  the  depth  of 
the  defense.  Given  the  tactical  situation,  what  technique  would  best  assure  that  Lane  Alice  is 
closed  when  required?  (Circle  all  that  apply.) 

[Question  20a:  NPE  90.5  %  correct,  PE  86.7  %  correct] 

[Question  20b:  NPE  81.0  %  correct,  PE  73.3  %  correct] 

[Question  20c:  NPE  97.6  %  correct,  PE  83.3  %  correct] 

[Question  20d:  NPE  83.3  %  correct,  PE  76.7  %  correct] 

A.  Establish  a  date  and  time  for  closure. 

B.  Transmit  a  code  word  from  the  CAB  Main  CP  to  direct  closure. 

C.  Designate  the  senior  engineer  in  the  area  as  the  firing  commander  and  have  him  execute 
closure  when  he  determines  the  enemy  is  near. 

D.  Authorize  the  C  Company  Commander  to  execute  closure  on  his  initiative  or  after 
predetermined  criteria. 
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Appendix  E 
Descriptives 


Table  E-l 

Commissioning  Source 


Prior  En 

listed 

Non-prior 

Enlisted 

Value 

Frequency 

Percent 

Frequency 

Percent 

ROTC 

7 

22.6 

18 

43.90 

USMA 

2 

6.5 

13 

31.70 

OCS 

20 

64.5 

10 

24.4 

Other 

2 

6.5 

1 

2.4 

Total 

31 

100 

42 

100 

Table  E-2 

Undergraduate  Major 


Prior  Enlisted 

Non-prior  Enlisted 

Value 

Frequency 

Percent 

Frequency 

Percent 

Business 

8 

27.6 

8 

19.5 

Engineering 

4 

13.8 

11 

26.8 

Humanities 

13 

44.8 

19 

46.3 

Science 

4 

13.8 

3 

7.3 

Total 

29* 

100 

41** 

100 

Note.  *2  participants  did  not  respond.  **  1  participant  did  not  respond. 
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Table  E-3 


Defensive  Planning  Prior  Knowledge  Test  Item  Statistics 


Q.  No. 

Mean 

SD 

Q.  No. 

Mean 

SD 

1 

.63 

.49 

lib 

.64 

.48 

2 

.44 

.50 

lie 

.85 

.36 

3a 

.65 

.48 

lid 

.50 

.50 

3b 

.33 

.47 

lie 

.78 

.42 

3c 

.41 

.50 

12a 

.86 

.35 

3d 

.29 

.46 

12b 

.86 

.35 

4a 

.56 

.50 

12c 

.58 

.50 

4b 

.97 

.16 

12d 

.99 

.11 

4c 

.78 

.42 

12e 

.65 

.48 

4d 

.51 

.50 

12f 

.99 

.11 

4e 

.11 

.32 

12* 

.96 

.19 

4f 

.87 

.34 

13a 

.60 

.49 

5a 

.40 

.49 

13b 

.67 

.47 

5b 

.68 

.47 

13c 

.79 

.40 

5c 

.90 

.31 

13d 

.40 

.49 

6 

.45 

.50 

14a 

.77 

.42 

7a 

.74 

.44 

14b 

.54 

.50 

7b 

.46 

.50 

14c 

.92 

.27 

7c 

.67 

.47 

14d 

.51 

.50 

7d 

.53 

.50 

15 

.67 

.47 

8a 

.58 

.50 

16 

.05 

.22 

8b 

.99 

.11 

17 

.33. 

.07 

8c 

.51 

.50 

18a 

.95 

.22 

8d 

.35 

.48 

18b 

.85 

.36 

8e 

.55 

.50 

18c 

.87 

.34 

8f 

.72 

.45 

18d 

.44 

.50 

8g 

.74 

.44 

18e 

.81 

.40 

9 

.54 

.50 

18f 

.67 

.47 

10a 

.68 

.47 

19 

.54 

.50 

10b 

.86 

.35 

20a 

.88 

.32 

10c 

.40 

.49 

20b 

.78 

.42 

lOd 

.62 

.49 

20c 

.92 

.27 

lOe 

.81 

.40 

20d 

.81 

.40 

11a 

.67 

.47 

Note.  N=  78  respondents. 

Mean  =  Percent  correct  responses. 
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Table  E-4 

Defensive  Planning  Prior  Knowledge  Test  Item  Difficulties  for  Non-prior  Enlisted  Officers 


Question 

Number 

%  Correct 
Responses 

Question 

Number 

(Cont.) 

%  Correct 
Responses 
(Cont.) 

Question 

Number 

(Cont.) 

%  Correct 
Responses 
(Cont.) 

1 

100 

20b 

81 

3c 

50 

8b 

100 

12a 

78.6 

4d 

50 

12f 

100 

7a 

76.2 

8c 

50 

4b 

97.6 

3a 

73.8 

9 

50 

12d 

97.6 

4c 

73.8 

10a 

50 

14c 

97.6 

8g 

73.8 

lid 

47.6 

20c 

97.6 

lib 

71.4 

12c 

47.6 

12g 

95.2 

18f 

71.4 

5a 

45.2 

18a 

95.2 

8f 

69 

7b 

45.2 

5c 

92.9 

13b 

69 

14d 

45.2 

4f 

90.5 

5b 

66.7 

6 

42.9 

18b 

90.5 

11a 

66.7 

18d 

42.9 

20a 

90.5 

7c 

64.3 

3b 

40.5 

lOe 

88.1 

lOd 

64.3 

8d 

38.1 

10b 

85.7 

13a 

64.3 

10c 

38.1 

lie 

85.7 

15 

64.3 

13d 

38.1 

13c 

85.7 

7d 

59.5 

2 

33.3 

12b 

83.3 

12e 

59.5 

17 

31 

18c 

83.3 

4a 

57.1 

3d 

23.8 

20d 

83.3 

8a 

57.1 

4e 

9.5 

lie 

81 

14b 

57.1 

16 

0 

14a 

81 

8e 

52.4 

18e 

81 

19 

52.4 

Note.  N=42  respondents  (Non-prior  enlisted  officers). 
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